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to  derive  world-space  coordinates).  This  approach  also  facilitates  robot  task 
training  by  a  "teach-by-showing"  strategy  for  specification  of  the  control  system 
reference  signal  commands.  The  dissertation  includes  an  in-depth  analysis,  design, 
and  evaluation  of  image-based  control. 

Image-based  visual  servo  (IBVS)  control  presents  formidable  engineering  problems 
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Abstract 


The  objective  of  this  dissertation  is  to  develop  analytical  tools  for  the  design  and  evaluation  of 
feedback  compensators  for  dynamic  vision-based  robot  control. 

Sensory  systems,  such  as  computer  vision,  can  be  used  to  measure  relative  robot  end-effector  (or 
tool)  positions  to  derive  feedback  signals  for  control  of  end-effector  positioning.  The  role  of  vision  as 
the  feedback  transducer  affects  closed-loop  dynamics,  and  a  visual  feedback  compensator  is 
required.  Vision-based  robot  control  research  has  focused  on  vision  processing  issues  (e.g.,  image 
filtering,  structured  lighting  environments,  and  image  feature  interpretation),  while  control  system 
design  has  been  limited  to  ad-hoc  strategies  or  linear  single  degree-of-freedom  (DOF)  systems.  This 
dissertation  formalizes  an  analytical  approach  to  dynamic  robot  visual  servo  control  systems  by 
organizing  and  categorizing  visual  control  strategies  into  position-based  structures  and  image-based 
structures.  The  image-based  structure  represents  a  new  approach  to  visual  servo  control,  which  uses 
image  features  (e.g.,  image  areas,  and  centroids)  as  feedback  control  signals,  thus  eliminating  a 
complex  interpretation  step  (i.e.,  interpretation  of  image  features  to  derive  world-space  coordinates). 
This  approach  also  facilitates  robot  task  training  by  a  "teach -by-showing"  strategy  for  specification  of 
the  control  system  reference  signal  commands.  The  dissertation  includes  an  in-depth  analysis, 
design,  and  evaluation  of  image-based  control.  \  -  -  -  — - ' 

Image-fcased  visual  servo  (IBVS)  control  presents  formidable  engineering  problems  for 
compensator  design,  including  coupled  and  nonlinear  dynamics,  kinematics,  and  feedback  gains, 
unknown  parameters,  and  measurement  noise  and  delays.  To  satisfy  these  requirements,  a  model 
reference  adaptive  control  (MRAC)  feedback  system  and  a  fixed  linear  model  following  controller 
(LMFC)  are  designed  and  evaluated  using  mainframe  simulation  models  of  two  and  three  DOF 
systems,  incorporating  both  nonlinear  kinematics  and  dynamics.  The  MRAC  is  found  to  provide 
superior  performance  to  the  LMFC  for  large-signal  trajectories  of  systems  with  unknown  or  nonlinear 
feature  sensitivity  gains.  The  issue  of  selecting  features  for  image-based  control  is  addressed  by 
defining  and  evaluating  an  index  of  feature  coupling.  The  dissertation  focuses  on  compensator 
designs  based  on  multiple  uncoupled  single-input  single-cutput  (SISO)  controllers,  for  coupled 
systems,  to  simplify  the  design  complexity  and  implementation. 


3 


Chapter  1 
Introduction 


1.1.  Overview 

The  purpose  of  this  chapter  is  to  present  an  overview  of  the  dissertation.  In  Section  1.2,  the 
background  and  problems  of  dynamic  visual  servo  control  of  robots  are  outlined,  and  a  novel 
image-based  approach  to  visual  servo  control  is  introduced.  Section  1.3  includes  a  summary 
of  the  dissertation  goals  and  design  approach  as  well  as  a  chapter-by-chapter  outline. 
Detailed  background  and  literature  surveys  are  included  in  Chapter  2,  in  which  the  problem 
areas  and  design  approaches  are  described  in  detail. 

1 .2.  Introduction  to  Visual  Servo  Control 

A  conventional  robot  arm  which  is  programmed  to  pick  up  an  object  from  a  table  must  be 
"taught"  the  position  and  orientation  of  that  object  before  it  can  carry  out  the  task.  This 
process  of  teaching  a  robot  positions  and  updating  them  for  each  new  object  or  location  is  a 
routine  part  of  current  robot  system  development,  and  has  become  a  major  limitation  in  the 
adaptation  of  robots  to  unstructured  tasks  including  many  routine  industrial  applications. 
Flexible  robotic  systems  would  provide  a  capability  to  automatically  modify  positions  and 
trajectories  to  accommodate  changes  in  the  task  or  task  environment.  Such  flexibility  is 
achievable  only  through  the  integration  of  sensors  into  robotic  systems,  and  visual  sensing,  in 
particular,  would  appear  to  provide  fundamental  capabilities  in  the  adaptation  of  robots  to 
unstructured  tasks.  This  dissertation  will  analyze  and  evaluate  strategies  which  integrate 
visual  sensing  with  robot  movements,  and  will  emphasize  the  design  of  such  "visual  servo 
control  systems"  in  order  to  achieve  stable  and  predictable  system  dynamics. 

Figure  1-1  illustrates  a  visual  servo  control  application  for  a  robot.  An  object  is  placed  on  the 
table  in  a  position  which  is  unknown  to  the  robot.  The  robot  has  not  been  preprogrammed 
with  knowledge  of  the  object  position.  In  this  sense,  the  task  environment  is  said  to  be 


Figu  re  1  •  1 :  Robot  Acquiring  Object  Using  Computer  Vision 
"unstructured".  A  television  camera  is  attached  to  the  robot  arm  and  provides  visual  sensing 
capability.  The  image  acquired  by  the  camera  must  be  processed  by  a  computer  vision  system 
in  order  to  identify  the  object  in  the  picture.  The  vision  system  extracts  information  from  the 
image  and  may  infer  relationships  between  the  spatial  position  of  the  object  and  camera 
position.  Such  relative  position  information  may  be  used  to  guide  the  robot  to  acquire  the 
object  from  the  table. 

Figure  1-2  shows  a  second  typical  application  of  visual  servo  control.  In  this  case,  a 
stationary  camera  is  used  to  observe  the  lead  tip  of  an  electronic  component  which  is  held  by 
the  robot  hand.  The  exact  position  of  the  lead  tip  relative  to  the  robot  hand  is  unknown  due  to 
the  uncertainty  in  the  forming  of  the  component  leads.  The  camera  also  observes  the  hole  in 
the  printed  circuit  board  where  the  lead  will  be  inserted.  The  vision  system  is  used  to  resolve 
the  uncertainty  in  lead  position  relative  to  the  hole  position.  The  robot  controller  then 
repositions  the  robot  arm  to  correctly  align  the  lead  with  the  hole. 


Several  key  issues  arise  in  the  design  of  visual  servo  control  systems  such  as  those 
described  above.  Processing  and  interpretation  of  image  information  to  infer  object  position 


Figure  1  -2:  Robot  Assembling  PC  Boards  Using  Computer  Vision 
and  orientation  are  difficult  and  computationally  complex  tasks.  The  resulting  estimates  may 
be  uncertain  and  require  computational  time  delays.  The  robot  controller,  in  turn,  must 
compensate  for  these  properties  of  the  vision  system  as  well  as  those  of  the  robot.  The 
resulting  integrated  system  should  have  dynamic  properties  which  are  predictable  and  stable 
even  in  the  face  of  unknown  task  conditions.  In  this  dissertation,  an  adaptive  controller  is 
proposed  to  provide  this  consistent  dynamic  system  response  under  unknown  conditions. 

The  interpretation  of  image  information  to  estimate  object  position  normally  requires  the 
extraction  of  quantitative  image  features  which  are  related  to  a  stored  object  model.  The 
resulting  position  estimates  may  be  used  to  design  a  "position-based"  visual  servo  control 
system.  In  this  dissertation,  an  alternative  strategy  is  explored.  Image  features  which  are 
monotonically  related  to  spatial  position  may  be  used  as  a  basis  for  control  in  place  of 
position  estimates.  Such  an  "image-based"  visual  servo  control  strategy  may  offer 
advantages  in  terms  of  reduced  delay  and  estimation  noise,  but  require  special  attention  to 
control  strategies  which  accommodate  position- to- feature  relationships.  The  image-based 
visual  servo  control  strategy  is  formalized  for  the  first  time  in  this  dissertation  and  control 
strategies  are  examined  in  detail. 
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Robot  Model 


or  Feedback) 


Figure  1  -3:  Basic  Robot  Positioning  Control 

In  order  to  understand  the  relationship  between  conventional  robot  position  control  and 
visual  servo  control,  it  is  useful  to  examine  a  robot  positioning  system  in  a  bit  more  detail. 
Such  a  system  may  be  described  by  the  block  diagram  in  Figure  1-3.  The  desired  position  of 
the  robot  hand  or  end-effector  is  specified  by  the  reference  position  vector  ,XREr  The  robot 
moves  by  displacement  of  individual  joint  angles  described  by  the  vector  £pEF,  and  a  model  of 
the  robot  geometry,  called  the  kinematic  arm  solution,  must  be  used  to  find  Q^ef  when  XpEF  is 
specified.  An  approximate  kinematic  arm  solution  is  always  available  for  a  real  robot  though 
variations  between  individual  robots  may  result  in  significant  inaccuracies  of  final  arm 
positioning. 

The  desired  joint  angles  described  by  £pEF  are  expressed  as  input  reference  signals  to 
individual  joint  angle  control  systems.  Each  joint  controller  uses  position  feedback  to 
accurately  position  the  joint  motor  to  the  desired  angle.  The  resulting  vector  of  joint  angles  £ 
is  therefore  obtained  by  independent  closed-loop  control  of  the  joints  with  respect  to  the 
desired  £REF- 

The  position  of  the  robot  end-effector  when  the  joints  are  set  at  angles  £  is  determined  by 
the  robot  geometry.  In  general,  the  robot  geometry  may  be  slightly  different  than  the  robot 
model,  and  therefore  the  actual  end-effector  position  £  may  differ  from  the  desired  position 
XREF.  In  real  systems,  there  is  no  way  to  determine  this  final  positioning  error  since  no 
measurement  or  feedback  of  final  end-effector  position  is  available.  In  this  sense,  the  control 


.  \ 
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loop  is  never  closed  around  the  end-effector  position  itself  leading  to  inherent  limitations  in 
the  ability  of  such  a  system  to  compensate  for  inaccurate  modeling  of  the  arm  or  to  derive 
positioning  error  signals  relative  to  unstructured  environments. 

The  addition  of  visual  sensing  as  a  measurement  system  for  relative  end-effector  position 
provides  a  basis  for  overcoming  some  of  these  limitations.  However,  the  use  of  computer 
vision  to  infer  position  and  orientation  of  objects,  or  interpret  general  three-dimensional 
relationships  in  a  scene,  is  in  general  a  complex  task  requiring  extensive  computing 
resources.  Techniques  which  may  exploit  simplier  sensors  or  minimize  processing  for  image 
interpretation  offer  major  advantages  for  visual  servo  control  implementations. 

Figure  1-4  illustrates  schematically  some  approaches  to  the  interpretation  of  a  two- 
dimensional  image  for  inference  of  three-dimensional  position  and  orientation.  In  the  block 
diagram,  a  sensor,  such  as  a  TV  camera,  is  used  to  acquire  a  two-dimensional  array  of 
brightness  values  from  the  three-dimensional  scene.  This  image  array  may  undergo  various 
types  of  computer  processing  to  enhance  image  properties  and  enable  the  extraction  of  both 
local  and  global  image  features.  These  features  typically  include  structural  components  of  the 
image  such  as  points,  lines,  and  areas,  as  well  as  quantitative  parameters  attached  to  them.  In 
reality  there  is  a  continuum  of  possible  image  features  and  their  available  transformations, 
and  their  choice  depends  on  the  purpose  and  requirements  of  their  subsequent  use  or 
interpretation.  The  relationship  between  such  structural  components  or  image  features  is 
then  used  to  interpret  the  observed  scene.  The  resulting  interpretation  provides  an  estimate  of 

A 

the  relative  position  X  which  may  be  used  for  visual  servo  control.  Such  position  estimates 
may  contain  object  model  inaccuracies  as  well  as  feature  measurement  inaccuracies  and 
therefore  may  introduce  measurement  noise  into  a  visual  feedback  control  system. 

Figure  1-5  shows  a  simple  example  of  a  visual  servo  control  structure.  This  system  is  called 
a  "static  look  and  move"  structure  for  visual  servo  control  and  is  described  in  more  detail  in 
Chapter  2.  The  system  operation  consists  of  a  sequence  of  independent  steps: 

I.Step  1:  The  vision  system  "looks"  at  the  scene,  or  object,  and  estimates  the 
end-effector  position  X; 

2.  Step  2:  The  position  estimate  is  sent  to  a  task  computer.  The  task  computer 
computes  the  difference,  AX.,  between  where  the  end-effector  should  be,  X.REF, 
and  the  current  position  estimate.  The  task  computer  then  issues  a  command  to 
the  robot  positioning  system  to  "move"  by  the  incremental  distance  A.&;  and 

3.  Step  3:  The  robot  moves  to  the  new  position.  Step  1  is  not  repeated  until  the  robot 
completes  the  motion  specified  by  the  "move"  command. 


Figure  1*4:  Computer  Vision 


STEP  2: 

"MOVE”  STEP  3: 


Figure  1-5:  Static  "Look  and  Move"  Control 

The  sequence  of  operations  is  repeated  until  a  specified  accuracy  is  achieved,  that  is,  AX.  is 
reduced  to  some  small  quantity.  The  number  of  iterations  will  depend  upon  the  particular  task. 
The  "static  look  and  move"  structure  demonstrates  the  concept  of  interactive  sensing  for 
robot  positioning,  but  is  not  a  dynamic  control  system  since  each  step  is  executed 
independently  and  in  sequence. 


Figure  1-6:  Dynamic  Visual  3ervo  Control 

If  the  visual  feedback  system  is  structured  so  that  the  three  steps  outlined  above  are 

A 

executed  in  parallel  (i.e.,  positions  estimates,  X,  and  position  errors,  AX.,  are  updated  as  fast 


as  they  are  measured,  and  position  corrections  are  commanded  to  the  robot  while  it  is 
moving),  then  dynamic  visual  servo  control  systems  such  as  that  illustrated  in  Figure  16  can 
be  synthesized.  The  role  of  computer  vision  as  the  feedback  transducer  now  affects  the 
over  all,  or  closed-loop,  system  dynamics,  and  a  visual  feedback  controller  is  required  for 
stability  and  acceptable  transient  response.  While  dynamic  control  has  the  potential  to 
achieve  faster  responses  than  "static"  systems,  dynamic  visual  control  presents  a  variety  of 
difficult  design  problems  which  are  not  currently  addressed  in  the  literature.  This  dissertation 
focuses  on  two  of  these  issues:  the  use  of  image  features  as  feedback  parameters  in  "image- 
based"  visual  servo  control,  and  the  implementation  of  adaptive  control  strategies  for  visual 
servo  control. 

In  image  based  visual  servo  (IBVS)  control  ,  the  image  features,  as  described  in  Figure  1-4, 
are  used  to  control  robot  motion  directly  without  inferring  position  in  the  spatial  coordinate 
f-ame.  This  approach  relies  on  the  systematic  variation  of  image  features  with  relative  object 
position.  Such  a  relationship  is  illustrated  in  Figure  1-7.  In  this  case,  the  task  is  specified  by 
portraying  the  current  camera  image  and  the  desired  camera  image.  The  image  features 
corresponding  to  an  image  trajectory  as  the  camera  moves  may  be  plotted  as  shown  in  the 
figure.  Monotonic,  although  nonlinear,  feature-to-position  realtions  are  obtained  in  this  case 
using  area,  relative  area,  and  center-of-gravity  as  features. 

In  an  IBVS  system,  the  reference  and  feedback  signals  are  defined  in  terms  of  these  image 
feature  values  which  correspond  to  current  and  desired  robot  positions.  The  feature  error 
may  be  derived  at  every  measurement  time  and  used  to  drive  the  robot  in  a  direction  which 
decreases  the  error.  The  subsystem  which  converts  the  feature  error  to  a  robot  control  signal 
comprises  the  heart  of  the  IBVS  controller.  This  dissertation  will  address  the  analysis  and 
design  of  such  controllers  for  IBVS  systems. 

The  IBVS  approach  to  visual  servo  control  presents  a  number  of  issues  which  require 
careful  formulation  and  analysis.  Selection  of  features  is  closely  tied  to  the  problem  of  image 
interpretation  since  features  which  are  closely  related  to  object  position  will  be  useful  for  both 
interpretation  and  servo  control.  Criteria  for  feature  selection  are  developed  in  this 
dissertation  based  on  feature  sensitivity  and  feature  coupling  measures.  The  feature 
sensitivity  (or  slope  of  the  feature-to-position  curve)  appears  in  the  control  structure  as  a 
small-signal  gain  term.  The  feature  sensitivities  are  typically  unknown  a’priori  and  change 
nonlinearly  with  robot  movements.  These  properties  of  the  feature  sensitivity  lead  to  our  use 
of  adaptive  control  strategies  in  this  study. 


Desired  Image 


Initial  Image 


Figure  1*7:  Features  Change  with  Motion 
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Since  an  IBVS  system  controls  image  feature  signals  rather  than  position  signals,  it  is 
difficult  to  design  a  system  with  precisely  predictable  position  trajectories.  The  relationship 
between  desired  feature  trajectory  and  resulting  position  trajectory  is  strongly  dependent  on 
the  choice  of  features.  An  important  result  of  the  simulation  studies  carried  out  in  this 
dissertation  is  the  demonstration  that  position  trajectories  are  typically  well-behaved  for 
smooth  monotonic  feature  sensitivity  relations. 

Flexible  robotic  systems  will  most  likely  incorporate  different  modes  of  control  for  different 
tasks  or  subtasks.  Visual  servo  control  is  one  such  mode  of  control  which  may  be  useful  in 
sensor-based  systems.  Position-based  and  image-based  visual  servo  controls,  in  turn, 
represent  alternative  but  complementary  strategies  within  this  framework.  As  pointed  out 
earlier,  there  is  a  continuum  of  features  which  may  be  derived  by  measurement  and 
transformation  from  an  image,  and  any  combination  of  these  may  be  useful  as  control 
parameters.  In  this  sense,  spatial  position  estimates  are  one  set  of  transformed  image 
features.  They  are  particularly  useful  when  available  and  reliable  since  an  approximate  model 
of  robot  geometry  is  typically  known.  The  use  of  alternative  features  in  the  IBVS  approach 
may  have  advantages  in  terms  of  speed  and  reliability  when  model  based  information  is  not 
available. 

In  more  general  terms,  IBVS  represents  a  case  study  of  robot  control  in  unstructured 
environments  where  model-based  information  is  either  unavailable  or  unreliable.  In  this 
sense,  the  adaptive  control  strategy  may  be  viewed  as  a  "learning  mode"  in  which  interactive 
sensing  of  the  environment  defines  an  updated  model  of  the  world  and  provides  a  basis  for 
consistent  control  strategies.  A  mobile  robot,  for  example,  consistently  must  predict  and 
verify  its  motion  using  unreliable  models  of  the  world  and  robot  itself.  The  identification  phase 
of  adaptive  control  provides  a  mechanism  to  test  and  update  dynamic  models,  and  the  image- 
based  approach  can  be  used  inversely  to  identify  and  confirm  image  features  based  on 
compatibility  of  predicted  and  observed  dynamic  changes. 

Another  example  of  adaptive  robot  control  using  sensors  arises  in  relation  to  force  and 
tactile  sensing.  Again,  a  consistent  model  relating  robot  control  commands  to  desired  force 
signals  may  exist,  and  performance  may  depend  on  identification  of  transformation  matrices 
and  on-line  synthesis  of  an  appropriate  controller.  Manipulation  of  structures  with  nonlinear 
compliant  elements  raises  exactly  these  questions. 

Visual  servo  control  is  a  fundamental  component  of  sensor- based  robotic  systems,  and  the 


analysis  and  design  of  systems  using  image  based  control  demonstrates  the  feasibility  of  a 
new  approach  which  may  provide  improved  performance  as  one  mode  of  an  integrated 
system.  Analysis  of  the  IBVS  problem  and  incorporation  of  adaptive  control  as  a  design 
solution  have  led  us  to  derive  principles  with  much  broader  applicability  than  to  visual  servo 
control.  An  increasing  number  of  problems  require  multiple  sensory  modalities  for  robot 
control  in  unstructured  environments,  and  adaptive  modes  of  interactive  sensing  will  become 
increasingly  important.  The  research  reported  in  this  disserta.lon  provides  a  basis  for 
exploration  of  many  such  adaptive  sensor-based  strategies. 


1 .3.  Outline  of  Dissertation 

This  section  presents  a  brief  summary  of  the  goals  and  design  approach  of  this  dissertation 
as  well  as  a  chapter-by  chapter  outline  of  the  technical  content.  A  table  of  acronyms  used  is 
included  as  Table  1-1  for  reference. 


1.3.1.  Dissertation  Goals 

The  goals  of  the  dissertation  are  to  address  the  problems  of  dynamic  visual  servo  control  of 
robots,  and  provide  insight  into  visual  servo  control  by: 

1.  Organizing  and  categorizing  visual  servo  control  systems  into  well-defined 
classical  feedback  control  structures.  Such  an  organization  facilitates  the 
analysis  and  design  of  a  servo  controller,  by  showing  the  role  of  computer  vision 
as  the  feedback  transducer,  and  by  accounting  for  the  static  and  dynamic 
properties  of  the  robot  and  measurement  systems,  which  must  be  compensated. 

The  categorization  can  also  serve  as  a  framework  for  comparative  evaluation  of 
different  control  approaches. 

2.  Analyzing,  designing,  and  evaluating  the  IBVS  approach  to  visual  servo  control. 

The  objectives  are  to  show  that  IBVS  can  produce  stable  systems,  with  good 
transient  responses,  and  suitable  path  performance:  and  to  provide  analytic  tools 
to  design  real  systems.  In  the  dissertation,  IBVS  systems  are  evaluated  using 
computer  simulation  models. 


In  the  next  section,  a  controller  design  approach  for  IBVS  control  is  proposed. 
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1.3.2.  Design  Approach 

Feedback  controllers  can  be  categorized  as  being  either  fixed  or  adaptive.  It  is  a  formidable 
engineering  task  to  design  effective  fixed  feedback  controllers  to  compensate  for  nonlinear 
and  unknown  systems.  The  gains  of  an  adaptive  controller  are  adjustable  and  are 
dynamically  changed  during  control  from  on  line  parameter  identification  of  the  unknown 
system.  The  model  reference  adaptive  control  (MRAC)  approach  has  been  extensively 
studied  for  the  control  of  unknown  linear  systems,  and  empirically  evaluated  for  dynamic 
nonlinear  systems.  Based  on  these  studies,  MRAC  appears  to  be  well  suited  for  IBVS  control, 
and  is  the  design  approach  developed  in  the  dissertation.  An  enhanced  MRAC  (with  series- 
parallel  reference  model  and  control  penalty)  is  selected  for  IBVS  control  to  suppress  noise, 
and  be  less  sensitive  to  control  signal  saturation.  In  the  dissertation,  the  author  modifies  the 
MRAC  to  account  for  measurement  delay,  and  identifies  the  requirement  of  a  control  penalty 
for  use  with  nonlinear  IBVS  systems.  The  dissertation  also  i  eludes  a  comparative  evaluation 
of  fixed  versus  adaptive  control.  The  fixed  controllers  are  synthesized  by  fixing  the  adjustable 
gains  of  the  adaptive  controller  (i.e.,  linear  model  following  control  (LMFC) ). 

Feedback  control  of  a  multi  input  multi-gutput  (MIMO)  system  (with  m  inputs  and  m  outputs) 
can  be  based  on  either  m  independent  §ingle-jnput  single -fintput  (SISO)  controllers,  or  on  a 
single  MIMO  controller  design.  MIMO  control,  of  coupled  systems,  seems  to  have  an 
inherently  greater  potential  to  achieve  the  desired  control  system  performance  criteria.  SISO 
controllers  are  simplier  to  implement,  since  they  are  computationally  less  complex,  and  can 
be  modularized.  A  modular  system  can  easily  be  extended  to  increasing  DOF,  and  distributed 
processing.  In  the  dissertation,  the  author  has  selected  to  investigate  an  SISO  approach, 
since  it  would  be  easier  to  implement  (in  future  research)  with  current  laboratory  and  factory 
computing  environments.  With  increasingly  powerful  and  less  expensive  processors,  an 
MIMO  approach  may  be  more  suitable  for  future  research  investigations. 

1 .3.3.  Outline  of  Dissertation 

In  Chapter  2,  visual  servo  control  is  discussed  in  detail.  The  chapter  begins  by  reviewing 
robot  kinematics,  dynamics,  and  control.  The  formal  categorization  and  organization  of 
previously  implemented  and  proposed  visual  servoing  strategies,  into  classical  feedback 
control  structures,  is  then  presented.  The  control  requirements  for  IBVS  are  described,  and 
the  MRAC  approach  is  proposed  to  meet  these  requirements. 
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In  Chapter  3,  the  enhanced  SISO  MRAC  algorithm  is  reviewed.  The  algorithm  is  modified  for 
systems  with  measurement  delay  and  coupling.  The  chapter  concludes  with  guidelines  for 
MRAC  parameter  selection. 

The  purpose  of  Chapter  4  is  to  specify  the  visual  servoing  tasks  configurations  (including  1, 
2,  and  3  DOF  systems)  which  are  evaluated,  in  simulations,  using  fixed  and  adaptive  IBVS 
control.  Mathematical  models,  required  for  the  computer  simulations  and  control  evaluations, 
are  developed,  and  incorporate  robot  kinematics  and  dynamics,  camera  picture  taking,  and 
feature  transformations. 

Chapters  5,  6  and  7  include  the  evaluation  of  LMFC  and  MRAC  control  of  progressively 
more  complex  systems.  The  complexity  relates  to 

•  DOF:  1 , 2,  and  3  DOF  (in  Chapters  5,  6,  and  7,  respectively); 

•  Dynamics:  linear  uncoupled  vs.  nonlinear  coupled; 

•  Kinematics:  linear  uncoupled  vs.  nonlinear  coupled;  and 

•  Features:  minimal  features  (i.e.,  the  number  of  features  available  for  control 
equals  the  number  of  DOF  to  control)  vs.  excessive  features  (i.e.,  feature 
selection  required  when  there  are  more  features  available  than  there  are  DOF  to 
control). 

The  evaluation  includes  a  sensitivity  analysis  of  the  feature  transformation  coupling; 
comparative  study  of  fixed  vs.  adaptive  control;  isolation  of  problem  areas  associated  with 
SISO  control  of  coupled  systems;  evaluation  of  a  path  constraint  strategy;  and,  evaluation  of  a 
feature  selection  strategy  to  discriminate  among  candidate  features. 

In  Chapter  8,  the  results  and  contributions  of  the  dissertation  are  summarized,  and 
directions  for  future  research  are  suggested.  Preliminary  investigations  of  MIMO  control, 
issues  of  physical  implementations,  and  simulated  control  of  a  5  DOF  IBVS  system,  are 
described. 

The  principal  contributions  of  the  dissertation  are: 

•  Formalization  of  dynamic  visual  servo  control  of  robots. 

•  Introduction  of  a  novel  "teach-by  showing”  image-based  control  approach. 

•  In-depth  analysis,  design,  and  evaluation  of  fixed  and  adaptive  control  of  IBVS 
systems. 


Extension  of  enhanced  SP  MRAC  to  control  of  systems  with  delay  and  nonlinear 
gains. 

Demonstration  of  enhanced  SP  MRAC  for  direct  joint- level  control  of  a  simulated 
articulated  robot  arm. 

Identification  of  the  applications  and  limitations  of  SISO  adaptive  control  of 
coupled  systems. 
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Chapter  2 

Visual  Servo  Control 


2.1 .  Overview 

The  objective  of  this  chapter  is  to  formalize  an  analysis  and  design  approach  for  dynamic 
robotic  control  using  computer  vision  for  sensory  feedback  information.  Controller  design  will 
focus  on  an  adaptive  approach  to  compensate  for  nonlinear  robot  dynamics,  and  nonlinear 
and  unknown  gains  introduced  by  the  sensory  feedback. 

This  chapter  is  organized  as  follows.  Control  system  analysis  and  design  requires  kinematic 
and  dynamic  models  of  the  robot  under  control.  In  Section  2.2,  manipulator  kinematics  and 
dynamics  are  reviewed.  Detailed  model  developments  for  specific  cases  are  presented  in 
Chapter  4.  In  Section  2.3,  a  review  of  control  is  presented,  describing  standard  non-sensor 
based  control  at  the  joint  and  world  levels  when  feedback  measurements  are  limited  to  joint 
level  space.  The  problem  of  controlling  highly  coupled,  nonlinear,  and  possibly  unknown 
dynamic  systems  is  described.  Control  law  approaches  for  these  systems,  including  model 
reference  adative  control  (MRAC),  are  discussed  since  similar  problems  arise  in  sensory 
based  systems. 

The  remainder  of  the  chapter  summarizes  the  contributions  of  the  dissertation.  In  Section 
2.4,  previously  implemented  and  proposed  visual  servoing  strategies  are  formally  organized 
and  categorized  into  classicial  feedback  control  structures.  Such  organization  has  not 
previously  appeared  in  the  literature.  While  the  scope  of  the  dissertation  is  limited  to  a 
detailed  design  and  evaluation  for  one  of  these  structures,  this  formalization  will  provide  a 
framework  for  future  investigations.  The  control  structures  are  based  on  feedback 
measurements  which  are  extended  to  include  world  coordinate  measurements.  Previously 
implemented  structures  derive  control  laws  based  on  relative  world  space  position 
information.  A  novel  structure  is  proposed,  which  uses  image  features  for  feedback.  The 
image-based  approach  has  potential  advantages,  including  being  able  to  teach  a  task  by 


20 


visually  "showing"  the  robot  what  to  do.  Design  of  the  image-based  visual  feedback 
controller  requires  compensation  of  robot  dynamics  and  kinematics,  and  of  the  feedback 
properties  including  delay,  coupling,  nonlinearity,  noise  and  unknown  parameters.  In  Section 
2.5,  an  adaptive  control  law  approach  is  proposed  to  meet  these  requirements.  The 
remainder  of  the  dissertation  includes  an  analysis,  design,  and  evaluation,  in  simulations,  of 
image-based  visual  servoing. 

2.2.  Robot  Kinematics  and  Dynamics 

A  robot  consists  of  a  sequence  of  links  connected  in  series  or  parallel  by  revolute  (i.e., 
rotational)  or  prismatic  (i.e  ,  translational)  joints  [Lee  82a].  Each  joint  is  independently 
actuated  by  electrical,  hydraulic,  or  pneumatic  devices  resulting  in  relative  motion  of  the  links. 
The  first  link  is  attached  to  a  reference  frame,  which  can  be  either  fixed  or  mobile,  while  the 
end  of  the  final  link  is  equipped  with  an  end-effector  (e.g.  mechanical  hand  or  tool)  to 
manipulate  objects  or  perform  assembly  tasks.  Each  joint- link  combination  constitutes  one 
degree-gf-freedom  (DOF).  For  general  purpose  tasks,  six  degrees-of- freedom  are  required; 
three  for  positioning  and  three  for  orienting.  Fewer  degrees-of-freedom  may  be  used  to 
accomplish  specific  tasks. 

A  block  diagram  of  an  N  DOF  robot  is  shown  in  Figure  2- 1 . 


Actuators 


Manipul ator 
Dynamics 


Geoinetry- 

Kinematics 


Figure  2-1:  Robot  Block  Diagram 

The  control  vector  y(t)  =  ,un(t)}T  represents  the  actuating  or  control  signals.  In 

response  to  u(t),  the  actuators  develop  forces  or  torques,  F(t),  which  drive  the  manipulator 
links.  The  resultant  joint  positions,  jg(t),  are  called  the  generalized  coordinates  [Paul  81].  Each 
joint  position,  q.(t),  is  referenced  to  the  coordinate  system  of  the  previous  link  in  the  chain. 
The  position  and  orientation  (or  pose)  of  the  end-effector,  £(t)  =  {x^tJ.-.X^t)}1,  is 
referenced  with  respect  to  a  world  coordinate  frame. 


2.2.1 .  Kinematics 


Robot  kinematics  deals  with  the  static  modeling  of  the  spatial  relationships  of  the  links  with 
respect  to  a  world  reference  frame  or  coordinate  system.  Robot  kinematics  thus  relates  joint 
variable  space,  g,  to  world  space  X .  World  space  can  be  defined  in  many  possible  coordinate 
systems  including  Cartesian,  spherical,  and  cylindrical.  While  typical  robotic  tasks  are  defined 
in  Cartesian  space,  particular  robot  geometries  may  lead  to  more  natural  definitions  in  other 
coordinate  frames. 

Kinematic  modeling  consists  of  two  problems: 

1 .  The  direct  kinematic  problem:  given  g,  find  X.,  and 

2.  The  inverse  kinematic  problem,  or  "arm-solution":  given  )(,  find  g. 

To  solve  these  problems,  kinematic  modeling  and  analysis  have  been  defined  in  terms  of 
homogeneous  transformation  matrices  [Paul  81].  A  brief  description  of  homogeneous 
transformations,  for  developing  robot  and  visual  servo  control  models,  is  presented  here. 
Homogeneous  transformation  matrices,  which  define  postions  and  orientation  of  coordinate 
systems,  have  the  following  important  property.  If  matrix  transformation  T./k  describes  the  i**1 
coordinate  frame  relative  to  the  klh  frame  and  Tj/i  describes  the  j,h  frame  with  respect  to  the  i,h 
frame,  then 

Tj/k  =  Tj/i  Ti/k 
and  consequently, 

T  =  T  1 T 

1  i/k  1  j/i  1  j/k 

For  robotic  manipulators,  a  Cartesian  coordinate  system  is  established  for  each  link 
according  to  the  systematic  convention  proposed  by  Denavit  and  Hartenberg  [Denavit  55]. 
The  homogeneous  transformation  matricies,  or  A  matricies,  describe  the  relative  translation 
and  rotation  between  successive  link  coordinate  frames.  Thus,  A1  describes  the  pose  of  the 
first  link  relative  to  the  reference  base,  and  A.  describes  the  pose  of  the  i,h  link  with  respect  to 
the  (i-1),h  link.  For  a  six  degree-of-freedom  manipulator, 

Tg  =  AiA2A3A4A5Ag  (2.1) 

where  T6  describes  the  pose  of  the  final  link  relative  to  the  world  reference  frame.  The  direct 
kinematic  problem  is  then  to  solve  for  g  according  to  (2.1).  The  inverse  "arm-solution"  is 
more  complex.  Methods  for  finding  g  given  Tg  are  presented  in  [Paul  81]. 

And  end -effector  (or  tool)  can  be  fixed  and  referenced  to  the  final  link  according  to  the 
transformation  matrix  Tt  When  X{oo(  is  the  tool  pose  relative  to  the  reference  frame,  then 
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T6  Ttool  “  "rtool/world_Hf-too|l 

where  H[]  maps  a  6  DOF  vector  into  the  equivalent  (4x4)  homogeneous  matrix  transformation. 

Thus,  the  Ts  matrix,  which  must  be  evaluated  by  the  "arm-solution",  becomes 

T  =T  1  T  (2  2) 

1 6  1  tool  '  tool/world  '  ' 

In  practice,  the  "arm-solution"  produces  only  estimates  of  the  joint  positions,  g,  which  are 
required  to  produce  a  desired  Tg.  The  following  error  sources  can  lead  to  inaccuracies: 

•  Inaccuracies  in  the  geometric  modeling  of  link  or  tool  dimensions,  or  tool 
mounting  dimensions; 

•  Unmodeled  link  compliance; 

•  Non-ideal  joint  coupling  between  links  such  as  gearing  backlash;  and 

•  Computer  quantization  and  round-off  in  the  "arm-solution"  evaluation. 

While  these  error  sources  can  be  reduced  by  careful  design  and  analysis,  they  can  deteriorate 
task  performance  as  the  required  accuracy  increases. 

2.2.2.  Rigid  Body  Dynamics 

Dynamic  modeling  of  the  robot  arm  is  required  for  designing  the  control  algorithm  to 
achieve  a  desired  system  performance.  Dynamic  equations  model  the  motion  of  the  robot 
links  in  generalized  coordinate  space.  This  motion  can  then  be  related  to  world  space  by  the 
kinematic  equations.  Several  approaches  are  available  for  formulation  of  the  arm 
dynamics  [Hollerbach  80].  The  Lagrangian  method  [Paul  81,Uicker  67],  which  is  reviewed 
below  and  derived  for  a  specific  case  in  Appendix  B,  has  the  advantages  that  it  is  a  systematic 
approach  (utilizing  the  A  matrices),  and  the  equations  have  easily  interpreted  physical 
meaning  which  have  significance  when  applied  to  control  law  design  (Section  2.3). 

The  Lagrangian  formulation  for  rigid  body  kinematic  linkeages  can  be  written  as 

D(q)fl  +  C(g,g)  +G(g)  =  F(t)  (2.3) 

where  D(g)  is  the  (n  x  n)  inertial  matrix,  C(g,g)  is  the  (n  x  1)  centrifugal  and  Coriolis  vector, 
G(g)  is  the  gravitaicnal  vector,  and  F(t)  are  the  joint  forces  or  torques.  Equation  (2.3)  signifies 
that  the  robot  dynamics  are  characterized  by  a  system  of  of  n  second  order,  coupled,  and 
nonlinear  differential  equations  whose  parameters  are  dependent  on  the  robot’s 
instantaneous  configuration  (generalized  coordinate  positions  and  velocities).  The  model 
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does  not  include  frictional  components,  which  may  be  significant  for  some  robot  designs. 
Viscous  friction  can  often  be  modeled  by  additional  linear  terms  of  the  form  Ffrjc(jon  =  KfrictjonQ- 
It  is  more  difficult  to  derive  simple  models  for  Coulomb  frictions.  Whether  or  not  they  are 
included  in  the  dynamic  model,  the  control  engineer  must  be  aware  of  their  effects. 

Independent  actuators  develop  the  required  force  or  torque  to  cause  link  motion.  While  the 
actuator  can  often  be  modeled  by  a  linear  system  when  operating  in  nominal  regions,  the 
control  engineer  must  be  aware  of  the  nonlinearities  and  time-varying  phenomena  in  the 
device.  For  example,  DC  motors  have  torque  offset  nonlinearities  when  opiating  in  low 
torque  regions.  Power  amplifiers  exhibit  current  limiting  saturation  nonlinearities,  and  the 
gain  drifts  with  time  and  temperature.  While  the  control  law  design  is  based  on  the  linear 
actuator  model,  the  control  engineer  must  choose  a  design  which  can  still  perform  adequately 
in  nonlinear  regions  and  be  relatively  insensitive  to  parameter  variations. 

2.3.  Control 

Robot  tasks  (e.g.,  put  the  peg  in  the  hole)  must  ultimately  be  defined  in  terms  of  a  set  of 
reference  or  command  signal  inputs  to  a  control  system.  The  control  problem  is  to  design  a 
control  law,  based  on  dynamic  models,  so  that  the  system  is  stable  and  the  output  tracks  the 
reference  signal  according  to  a  predefined  performance  specification. 

With  respect  to  robot  positioning  control,  two  levels  of  tasks  can  be  identified: 

1 .  Dynamic  joint  control,  and 

2.  Kinematic  (or  end-effector  positioning),  path  and  trajectory  control 

While  joint  control  deals  directly  with  the  issue  of  plant  dynamics  and  control  law  design  for 
reference  signal  tracking,  kinematic  control  deals  with  generation  of  the  reference  signal 
required  for  end -effector  positioning  and  motion  along  a  specified  path  or  trajectory. 
Depending  on  the  approach  to  control  law  design,  dynamic  and  kinematic  control  can  be 
coupled  or  treated  as  independent  problems. 

In  robotic  control  applications,  the  reference  signal  can  be  specified  in  terms  of  the 
generalized  coordinate  or  world  coordinate  parameters,  depending  upon  the  required  task. 
For  example,  when  a  robot  is  taught  a  desired  position,  or  trajectory,  through  the  "teach- 
mode"  operation  [Luh  83],  joint  positions  are  directly  recorded  and  can  be  "played-back"  as 
the  desired  reference  signals.  If  the  task  requires  that  the  end-effector  move  in  a  straight  line 


path  to  the  end  point,  then  the  reference  signal  must  first  be  specified  in  Cartesian  space. 
Without  sensor-based  control,  with  which  the  position  of  the  end-effector  can  be  measured 
directly,  controller  designs  are  constrained  to  use  feedback  measurements  limited  to  the 
generalized  coordinates  (i.e.,  joint  displacements).  Thus,  reference  signals  defined  in 
Cartesian  space  must  be  mapped  into  estimated  joint  level  reference  signals  by  the  inverse 
kinematic  "arm-solution."  This  approach  results  in  open-loop  control  of  the  end-effector. 

This  section  includes  a  review  of  control  at  the  joint  and  world  levels  when  the  feedback 
measurements  are  limited  to  the  generalized  coordinate  space.  In  Section  2.4,  visual  servo 
control  structures  are  described  which  extend  the  measurement  space  to  include  world 
coordinates. 


2.3.1 .  Joint  Level  Control 


Practical  approaches  to  control  law  design  for  robotic  manipulators  include: 

1.  Feed-forward  open-loop  control  [Lee  82b,  Luh  80,  Hollerbach  80,  Raibert 
78,  Albus  75]; 

2.  Linear  servo  control  [Paul  81 ,  Luh  83,  Williams  83];  and 

3.  Adaptive  servo  control  [Chung  82,  Dubowsky  79,  Horowitz  80,  Koivo  80,  Koivo 
83,  LeBorgne  81], 

These  controllers  can  be  applied  independently,  or  in  combination,  as  each  has  certain 
advantages  and  disadvantages. 


A  block  diagram  of  a  robot  under  computer  control  is  shown  in  Figure  2-2.  The  control 
system  represents  a  sampled-data  system  in  which  u(k)=u(kT),  where  T  is  the  constant 
sampling  period.  The  control  signals  are  applied  through  gigital-to-gnalog  converters  (DAC) 
which  can  be  modeled  by  the  cascade  of  an  ideal  impulse  sampler  and  a  zero-order  hold  with 
saturation  nonlinearity  [Neuman  79a].  Over  the  interval  kT  <  t  <  (k+  1)T, 


ua(t)  = 


Uj(kT) 


sat 


+  V 
-V 


sat 


for  lUjfktJKV^, 
for  uj(kT)>V 
for  uj(kT)<-Vsat 


(2.4) 


where  Vsa(  is  the  maximum  DAC  output  signal  For  simplicity,  the  block  diagram  in  Figure 
2-2  shows  position  feedback.  The  following  discussions  do  not  preclude  the  use  of 
tachometers  and  observers  to  measure  and  estimate  joint  velocity  and  acceleration. 
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Figure  2-2:  Robot  Control  Block  Diagram 


2. 3. 1.1.  Feed-Forward  Open-Loop  Control 

In  feed-forward  open-loop  control,  the  reference  signals  are  specified  as  a  sequence  of  joint 
position,  velocity  and  acceleration  values  {s^kj.g^kj.g^k)}.  Feed-forward  open-loop 
controllers  use  the  nonlinear  dynamic  equations  of  motion  to  compute  the  joint  torques  or 
forces,  F(k),  to  track  the  reference  signals.  (In  practice  the  dynamic  equations  include  the 
actuator  dynamics  so  that  u(k)  is  computed).  Since  this  method  supplies  a  direct  solution  to 
these  equations,  it  has  the  advantage  of  precision  tracking  of  the  reference  commands.  It  has, 
however,  three  significant  disadvantages.  First,  this  open-loop  controller  is  sensitive  to 
parameter  variations  in  the  actuators,  robot,  and  DAC  converters,  and  to  unmeasurable 
external  torque  or  force  disturbances.  Second,  this  technique  is  computationally  intensive. 
The  on  line  computational  complexity  can  constrain  the  sampling  interval,  T,  thus  limiting  the 
maximum  allowed  joint  velocities.  While  the  Lagrange-Euler  approach  is  simple  and 
systematic,  it  is  inefficient  for  on-line  computation  and  control.  Wore  efficient  algorithms  for 
on-line  computation  are  available.  These  include  recursive  Nev/ton-Euler  formulations  [Lee 


82b,  Luh  80],  recursive  Lagrangian  formulations  [Hollerbach  80],  and  table  look  up 
methods  [Raibert  78,  Albus  75].  Finally  the  computed  torques  are  only  as  accurate  as  the 
model  in  (2.3).  Uncertainties  in  the  model  parameters,  including  the  oavload.  and  phenomena 
which  are  difficult  to  model,  such  as  backlash,  Coulomb  friction,  voltage  offsets,  and  link- 
compliance,  will  introduce  deviations  from  the  reference  signals.  The  relative  mechanical 
simplicity  of  the  CMU  Direct-Drive  Arm  is  an  attempt  to  reduce  such  uncertainty  [Asada  82], 

To  compensate  for  tracking  errors  due  to  imprecise  models  and  controller  sensitivity  to 
parameter  variation  and  uncertainty,  these  controllers  can  be  augmented  by  conventional 
linear  feedback  controllers. 

2.3.1 .2.  Fixed  Linear  Servo  Control 

Linear  feedback  or  servo  controllers  compute  control  signals  from  the  servo  errors, 
including  both  position  errors  e(k)  and  velocity  errors.  In  fixed  control,  the  controller  gains 
are  constant. 

In  fixed  controller  design  synthesis,  it  is  assumed  that  the  robot  dynamics  are  time- invariant 
or  can  be  linearized  about  a  nominal  operating  point  or  trajectory.  Upon  examination  of  the 
dynamic  equations  (2.3),  the  robot  can  be  assumed  to  have  linear  dynamics  if  the  Coriolis- 
centrifugal  terms  are  neglected,  and  if  the  interial  matrix  and  gravitational  torques  are 
constant.  When  the  Coriolis-centrifugal  torque  components  are  neglected,  they  can  be 
viewed  as  external  torque  disturbances  which  must  be  rejected  by  the  feedback  control 
compensation.  At  low  operating  speeds,  the  magnitude  of  these  apparent  disturbances  are 
often  assumed  to  be  small.  However,  the  relative  contribution  of  these  terms  may  still  be 
significant  during  typical  movements  at  low  velocities.  If  the  actuators  are  coupled  to  the 
joints  by  gear-reducing  mechanisms,  then  the  Coriolis-centrifugal  forces  reflected  back  to  the 
actuator  are  reduced  by  the  gear  ratio.  In  present  industrial  manipulators,  gearing  is  used  to 
optimize  power  transfer  from  the  motor  to  the  link,  and  minimize  the  required  motor  size. 
Gravitational  forces  reflected  back  to  the  motor  are  divided  by  the  gear  ratio  and  reflected  link 
inertias  are  reduced  by  a  factor  of  the  gear  ratio  squared.  Thus,  apparent  variations  in  the 
inertial  and  gravitational  forces  are  reduced.  Under  the  assumptions  that  Coriolis-centrifugal 
torques  are  neglected,  and  inertial  and  gravitational  changes  remain  small,  the  control 
engineer  designs  a  fixed  controller  using  methods  derived  from  classical  and  modern  control 
engineering.  Ideally,  the  fixed  control  design  is  insensitive  to  small  parameter  variations  and 
small  external  torque  disturbance  sources.  In  practice,  it  is  a  formedable  engineering  task  to 
design  a  robust  fixed  controller  for  such  systems. 


Feedback  controller  designs  can  either  be  uncoupled  or  coupled.  Coupled,  or  multi-input 
multi  output  (MIMO),  control  laws  derive  u(k)  as  a  linear  combination  of  the  entire  reference 
and  feedback  vectors.  Independent  uncoupled,  or  gingle-input  gingle-gutput  (SISO), 
controllers  produce  Uj(k)  based  only  on  the  ith  reference  and  feedback  signals  (i.e.,  e((k) ).  For 
an  n  DOF  robot,  n  independent  SISO  controllers  are  used.  There  are  no  systematic  rules  for 
design  of  SISO  controllers  for  coupled  systems.  MIMO  control  design  is  more  straight¬ 
forward,  and  can,  in  theory,  achieve  a  larger  degree  of  dynamic  uncoupling  than  SISO 
control.  However,  SISO  controllers  facilitate  implementation  because  they  are: 

1.  Computationally  less  complex  than  a  coupled  controller; 

2.  Modular,  and  computation  can  be  distributed;  and 

3.  Simpler  to  tune  than  a  coupled  controller. 

Commercially  available  industrial  robots  use  SISO  controllers.  To  the  best  of  the  authors 
knowledge,  most  use  SISO  control  which  is  not  augmented  by  feedforward  control,  because 
of  gearing  and  their  lower  speed  and  tracking  precision  requirements. 

A  typical  linear  SISO  controller  is  the  Proportional  +  Summation  +  Difference  (PSD)  regulator 
derived  as 

k 

U,(k)  =  Kpe.(k)  +  Ks£  e.(k)  +  K0[e.(k)-ej(k-1)] 

is  1 

Fixed  controller  gains  are  selected  using  root-locus  or  frequency  response 
techniques  [Williams  83]  (and  ultimately  manually  tuned)  to  achieve  an  acceptable  trade-off 
between  transient  speed  of  response  and  steady-state  velocity  errors  to  ramp  inputs.  The 
gain  selection  process  is  based  be  on  a  nominal  robot  configuration  and  neglecting  inertial 
coupling.  It  has  been  suggested  that  improved  performance  results  if  a  gain-scheduling 
approach  is  utilized,  where  the  adjustable  fixed  gains  are  selected  as  a  function  of  the  robot 
configuration  [Paul  81]. 

Relative  to  feedforward  control,  fixed  linear  controllers  have  the  advantages  of  being 
simpler  to  implement  in  real-time  and  can  be  less  sensitive  to  parameter  variations  and 
uncertainty.  Fixed  controllers  cannot  achieve  adequate  dynamic  response  at  higher  speeds, 
and  over  a  broad  range  of  arm  configurations. 


2.3.1 .3.  Adaptive  Feedback  Control 

Recently  researchers  have  sought  a  controller  approach  which  can  compensate  for 
parameter  uncertainty  and  variation  over  a  wide  range,  while  operating  at  high  joint  speeds.  In 
addition,  the  controller  should  be  compatible  with  microprocessor  implementation  to  make  it 
a  feasible  solution  for  common  industrial  application.  To  this  end,  initial  research  has  been 
conducted,  based  on  computer  simulations,  into  the  application  of  adaptive  control  to  robotic 
manipulators  [Chung  82,  Dubowsky  79,  Horowitz  80,  Koivo  80,  Koivo  83,  LeBorgne  81].  While 
adaptive  control  theory  has  been  in  existence  for  twenty  years,  high  speed  digital  computing 
has  now  made  it  a  viable  option  for  on  line  robotic  control  applications.  Adaptive  controllers 
have  adjustable  gains  which  are  automatically  tuned,  on-line,  based  on  measurement  of 
system  performance  and/or  characteristics.  Modern  approaches  to  adaptive 
control  [Goodwin  84]  can  be  categorized  as  either 

1 .  Self-Iuning  Regulators  (STR)  [Astrom  77],  or 

2.  Model  Reference  Adaptive  Control  (MRAC)  [Landau  79]. 

Both  approaches  derive  linear  models  of  the  manipulator,  whose  inadequacies  are 
compensated  by  permanent  parameter  adaptation  [LeBorgne  81].  While  both  STR  and  MRAC 
approaches  lead  to  similar  algorithms  [Shah  80],  the  MRAC  approach  provides  a  simple  and 
flexible  means  of  specifying  closed-loop  system  performance.  STR  and  MRAC  configuration 
block  diagrams  are  shown  in  Figures  2-3  and  2-4.  STR’s  derive  a  linear  controller,  whose 
gains  are  a  function  of  identified  robot  parameters,  under  the  assumption  that  the  robot  is 
linear  and  constant,  but  has  unknown  parameters,  An  on-line  identifier  estimates  parameters 
of  an  equivalent  input/gutput  (I/O)  linear  model  of  the  robot  based  on  the  I/O  information 
vectors  g(k)  and  g(k).  An  equivalent  I/O  model  predicts  the  output  g(k)  from  past  and  present 
I/O  information  independent  of  the  physical  model  of  the  robot.  The  estimated  parameter 
values  are  then  used  in  the  linear  controller  as  though  they  where  the  actual  parameters.  Both 
SISO  and  MIMO  equivalent  model  formulations  can  be  used.  Neuman  and  Stone  [Neuman 
83]  have  justified  the  latter  modeling  approach  by  demonstrating  that  individual  joints  of  a 
coupled  and  nonlinear  robot  can  be  modeled  by  linear  time-varying  second-order  SISO 
transfer  functions.  They  show  that  the  transfer  function  parameters  vary  smoothly  in  the  work 
space  as  a  function  of  the  joint  positions,  velocities,  and  accelerations. 

In  MRAC  control,  the  reference  model  output  gR(k)  specifies  a  stable  and  realizable 
closed  loop  dynamic  response  of  the  the  output  g(k)  to  the  reference  signal  aRe,(k).  The 
difference  between  the  reference  model  output  and  the  process  is  called  the  full-parallel  (FP) 
output  error: 
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Figure  2-4:  Model  Reference  Adaptive  Control 
e^k)=gR(k)-ci(k). 

The  adjustable  controller  utilizes  I/O  information  to  adjust  the  gains  on-line  to  drive  the  FP 
error  to  zero,  thus  forcing  the  robot  output  to  track  the  reference  signal  in  accordance  with 
the  performance  specified  by  the  model.  Within  the  framework  of  robot  control,  two  methods 
have  been  applied  [LeBorgne  81]: 

1.  Identification-error  method,  and 

2.  Output-error  method 
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In  the  identification-error  method  (which  is  developed  in  detail  in  Chapter  4),  an  identifier 
predicts  the  robot  joint  outputs,  t(k),  based  upon  parameter  estimates  of  an  equivalent  linear 
I/O  model.  The  identification  error 
g|D(k)  =  fl(k)-t(k) 

drives  the  adjustment  mechanism  which  updates  the  estimates  of  the  equivalent  parameters. 
These  estimates  are  then  used  to  adjust  the  gains  of  a  linear  controller  which  is  driven  by  the 
model  output.  In  the  output-error  method,  the  FP  error  drives  an  adjustment  mechanism 
which  directly  adjusts  the  controller  gains.  From  a  physical  implementation  point-of-view,  the 
identification  error  approach  has  the  advantage  that  the  adjustment  mechanism  need  not  be 
turned  off  during  control  signal  saturation  [Morris  79]. 

The  adjustment  mechanisms  of  STR  or  MRAC  controllers  can  be  designed  from  either 
parametric  optimization  or  stability  viewpoints.  Parametric  optimization  techniques  include 
recursive  least  squares  (RLS)  and  gradient  (or  "steepest-descent")  methods.  The  stability 
approaches  derive  adjustment  mechanisms  using  stability  theory  to  assure  that  the 
identification  .errors  are  asymptotically  stable.  Since  the  adjustment  mechanisms  are 
nonlinear,  either  Popov’s  hyperstabilty  or  Lyapunov’s  stability  criteria  have  been  used.  Both 
the  optimization  and  stability  approaches  make  the  assumption  that  the  system  under  control 
is  linear  and  time-invariant  (or  slowly  time-varying). 

For  the  optimization  approach,  stability  analysis  of  the  over  all  closed-loop  system  is 
demonstrated  through  the  theory  of  stochastic  convergence  of  optimization 
mechanisms  [Astrom  77].  When  hyperstabilty  or  Lyapunov  mechanisms  are  used,  stability  of 
the  identification  error  is  related  directly  to  the  FP  error.  Beacuse  of  the  assumptions  of 
linearity  and  time-invariance,  stability  of  adaptive  controllers  applied  to  nonlinear  and  time- 
varying  robot  dynamics  has  not  been  proven,  and  it  remains  an  open  question  as  to  whether  it 
can.  Control  engineers  rely  on  ad-hoc  assumptions  that  fast  adaptation  speeds  and  the  ability 
to  model  the  robot  as  an  equivalent  time-varying  linear  system  will  lead  to  stable  systems. 

STRs  and  MRACs  can  be  formulated  as  either  SISO  or  MIMO  controllers,  depending 
whether  the  robot  is  modeled  as  an  equivalent  SISO  or  a  MIMO  system.  The  adaptive 
controllers  can  be  used  independently  or  augmented  by  feedforward  control.  When  used 
independently,  minimal  a’priori  knowledge  of  the  dynamics  is  required. 

STRs  applied  to  multi-OOF  manipulators  have  been  reported  to  control  arm  simulations  over 
a  wide  range  of  speeds  and  configurations  [Chung  82,  Koivo  80,  Koivo  83,  LeBorgne  81]. 


31 


Koivo  and  Guo  [Koivo  83]  applied  both  coupled  and  uncoupled  STRs  (using  RLS)  to  control  a 
6  DOF  arm.  They  observed  that  the  MIMO  algorithm  required  a  longer  time  to  converge  and, 
at  least  in  tne  simulated  trajectories,  they  showed  no  improvement  in  response  relative  to  the 
SISO  controllers.  LeBorgne  [LeBorgne  81]  applied  SISO  STRs  and  output  error  MRACs 
(using  a  RLS  adjustment  mechanism)  to  control  of  a  3  DOF  arm,  and  found  that  the  STR  was 
computationally  less  complex,  but  that  the  MRAC  showed  better  overall  performance.  Chung 
and  Lee  [Chung  82]  use  feedforward  control  augmented  by  an  RLS  STR.  In  this  approach  the 
dynamic  robot  model  is  linearized  about  a  nominal  trajectory  to  obtain  a  small-signal 
perturbation  model.  The  adaptive  controller  then  identifies  a  small-signal  model  and  drives  the 
perturbations  to  zero. 

Successful  simulation  results  of  MRAC  control  have  also  been  reported  [Dubowsky 
79,  Horowitz  80,  LeBorgne  81].  Le  Borgne  [LeBorgne  81]  applied  SISO  identification  error 
and  output  error  methods  (using  RLS  adjustment  mechanisms)  and  observed  that  the 
identification  error  approach  displayed  better  overall  performance.  Neuman  and 
Morris  [Neuman  80,  Neuman  79b]  performed  extensive  analysis  and  evaluation  of  uncoupled 
MRAC  applied  to  linear  SISO  plants,  with  emphasis  on  implementation  and  computational 
complexities.  They  suggest  that  these  uncoupled  algorithms  can  also  be  applied  to  robotic 
control  [Morris  81].  Enhancements  to  the  basic  MRAC  algorithm  are  included  to  reduce  the 
effects  of  controller  saturation  and  measurement  noise.  Added  stability  constraints  limit  the 
adjustable  controller  gains  to  assure  a  bounded  and  stable  controller.  They  conclude  that  an 
enhanced  identification-error  MRAC,  with  a  hyperstable  adjustment  mechanism,  produce  the 
best  results  with  respect  to  convergence  speeds,  noise  supression,  recovery  from  control 
signal  saturation,  estimated  parameter  accuracy,  and  propagation  delays  [Morris  81]. 

2.3.2.  Kinematic  Control 

Robotic  positioning  tasks  require  that  the  end-effector  move  between  specified  points  in 
space,  and  along  specified  paths  and  trajectories.  This  kinematic  control  problem  can  be 
separated  from  the  dynamic  control  problem  if  it  can  be  assumed  that  the  joint  level 
controllers  can  assure  reference  signal  tracking.  Other  approaches,  including  optimal  and 
adaptive  control,  combined  the  kinematic  and  dynamic  control  problem.  Kahn  and 
Roth  [Kahn  71]  address  the  near-minimum-time  optimal  control  problem  for  a  nominal 
trajectory.  This  and  other  optimization  techniques  are  still  too  complex  to  be  practical.  More 
recently,  it  as  been  suggested  to  extend  joint  level  adaptive  controllers  to  world-level 
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tracking  [Koivo  83,  Takegaki  81].  Measurement  of  the  world  level  positions  must  still  be 
predicted  by  the  direct  kinematic  solution.  Takegaki  [Takegaki  81]  has  reported  effective 
performnace  of  an  output-error  type  controller  in  initial  simulation  studies.  In  practice,  the 
kinematic  problem  is  still  considered  to  be  independent  of  robot  dynamics. 

End-points  and  paths  for  the  end-effector  can  be  specified  in  three  ways: 

1.  Teach-mode1  (points  defined  at  joint-level); 

2.  Pre  defined  task  (points  generally  defined  in  world  space);  and 

3.  On  line  decisions  (information  supplied  by  external  sensors,  and  points  defined  in 
world  space). 

Points  defined  in  world  space  must  first  be  mapped  into  joint  space  using  the  "arm-solution". 
Control  of  the  world  reference  signal  Xref(k)  is  depicted  in  Figure  2-5,  where  the  closed-loop 
block  represents  the  joint-level  control  system  depicted  in  Figure  2-2. 


Figure  2-5:  Open-Loop  Kinematic  Control 


If  the  end -effector,  or  tool,  is  referenced  to  the  final  link  by  Ttoo|,  then  the  arm-solution  must 
evaluate  T0  according  to 

wk>=T;LHi*«ik>] 

A 

The  resultant  joint  level  reference  signal,  a,.e((k),  is  an  estimate  due  to  any  of  the 
aforementioned  inaccuracies  in  the  "arm-solution.”  In  real  systems,  there  is  no  way  to 
determine  the  final  positioning  error  since  no  measurement  or  feedback  of  final  end-effector 
position  is  available.  In  this  sense,  the  control  loop  is  never  closed  around  the  end-effector 
postion  itself  leading  to  inherent  limitations  in  the  ability  of  such  a  system  to  compensate  for 

Whe  "teach-mode"  operation,  for  programing  industrial  robots,  includes  manually  guiding  the  manipulator  to 
desired  locations  and  recording  the  joint  positions  at  each  point.  The  stored  points  can  then  be  used  to  derive  the 
reference  signals. 


inaccurate  modeling  of  the  arm  or  to  derive  positioning  error  signals  relative  to  unstructured 
environments. 


The  time-amplitude  profile  of  the  reference  signal  dictates  end-effector  path  motion 
between  task  points.  For  example,  when  a  well-defined  path  is  required  (e.g.,  to  avoid 
obstacles),  straight-line  motion  is  specified.  The  points  on  a  straight-line  (including 
orientations)  are  mapped  into  joint  level  commands  via  the  arm-solution.  To  compensate  for 
speed  limitations  in  computing  the  "arm-solution",  linear  interpolation  of  the  resultant  joint 
positions  are  used  to  supply  intermediate  points.  When  the  task  requires  that  the  robot 
cannot  stop  between  end-points  of  a  path  (e.g.,  welding),  trajectory  control  deals  with 
specification  of  smooth  transitions  between  path  segments.  One  approach  is  spline  function 
fitting  [Paul  81 1  to  the  endpoints  of  adjacent  path  segments. 

2. 3. 2.1 .  Incremental  Motion  Control 

Robotic  tasks  often  require  the  end-effector  to  move  by  increments,  AXtoo,,  rather  than  to 
absolute  positions.  If  an  absolute  positioning  control  system  is  used,  then  the  "arm-solution" 
must  evaluate  Tg  according  to 

T„  (k)  -  T  ,(k)  T.  .  T  (k)  T. 1  . 

6-rer  '  uv  '  iocl  me'  '  tool 

where  Tg(k)  is  the  estimated  present  position  transfonnation  based  on  the  direct  kinematic 
solution,  and  Tjnc=H[AXjnc]. 


A  more  efficient  approach  to  incremental  motion  is  resolved-rate-motion  (RRM) 
control  [Whitney  72].  In  this  approach  the  inverse  arm  Jacobian  is  used  to  find  the 
incremental  joint  velocities  which  are  required  to  achieve  motion  in  the  specified  directions.  If 
the  direct  kinematic  solution,  K,  maps  g  into  X  according  to  X  =  K(g),  where  K  can  be  derived 
from  Tg,  then  the  arm  Jacobian,  Jarm,  is  defined  by 


,  9K(fl) 

arm~  3g 


(2.5) 


World  increments  are  then  related  to  joint  increments  according  to 

«'Ja™5a 

and  the  joint  increments  are 
Sg  =  Ja'  S  X 

arm  — 

These  joint  increments  then  drive  closed-loop  velocity  servo  mechanisms. 


An  incremental  robot  system,  using  either  the  direct  "arm-solution"  or  the  RPM  approach,  is 
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Figure  2-6:  Incremental  Robot  System 

represented  by  the  block  in  Figure  2-6  for  reference  in  Section  2.4.  The  system  in  Figure 
2-6  includes  internal  joint  level  feedback  and  kinematic  decoupling.  Such  a  system  is  ideally 
linear  uncoupled,  without  discrete  time  delays;  i.e.,  each  DOF  can  be  represented  by  a 
transfer  function  of  the  form 

A*<S>,oo,. _ !_ 

X(S><ool-ReH  S<TS+1) 

The  next  section  extends  dynamic  robot  control  to  closed-loop  control  of  world-space 
coordinates. 

2.4.  Visual  Servo  Control  Structures 

Research  in  computerized  vision  for  automation  is  directed  toward  fast  systems  which  can 
interpret  image  features  for 

•  Object  recognition  [Myers  80,  Fu  82,  Vuylsteke  81]; 

•  Inspection  [Jarvis  80,  Porter  80];  and 

•  Measurement  of  object  positions  relative  to  the  vision  transducer,  or  relative 
positions  between  objects  [Hall  82,  Tani  77,  Birk  79,  Agin  79]. 

Such  capabilities  may  be  combined  into  more  general  modes  of  scene  interpretation. 

Complete  functional  systems  are  not  yet  available.  If  such  a  system  were  used  in  conjunction 

with  a  robotic  system,  then  direct  world  space  measurements  of  relative  end-effector 

positions  could  be  achieved.  For  example,  the  vision  transducer  can  be  fixed  to  the  end- 

effector  and  move  with  the  manipulator  as  it  observes  an  object's  pose  relative  to  itself  [Agin 

79];  or  equivalently,  it  can  be  stationary  while  observing  the  relative  pose  between  an  end- 

effector  and  object  [Ward  79].  Also,  the  transducer  can  be  stationary  while  observing  the 

relative  pose  between  itself,  or  equivalently  the  world,  and  an  object  being  manipulated  or  the 

end-effector  itself  [Birk  79].  These  integrated  systems  have  the  potential  for  improving  robot 

accuracy,  and  extend  the  robot's  capability  to  operate  in  unstructured  environments 

containing  objects  which  can  be  in  random  poses  or  possibly  moving. 


When  vision  systems  are  integrated  with  robots,  the  measurement  space  is  extended  to 
include  relative  the  end-effector  pose.  Positioning  control  systems,  which  directly  utilize  such 
measurements,  can  be  synthesized  using  visual  feedback  [Sanderson  83a].  The  resulting 
visual  servo  system  is  depicted  in  Figure  2-7,  where  3"ref  and  3* represent  the  reference  and 
feedback  signals. 


Figure  2-7:  Robotic  Visual  Servo  Control 

The  role  of  computer  vision  as  the  measurement  process  affects  the  overall  system  dynamics, 
and  a  visual  feedback  controller  is  required.  The  linearity,  noise  properties,  coupling,  and 
computational  delays  of  this  measurement  process  become  essential  considerations  for 
controller  design.  Formal  analysis  and  design  of  feedback  controllers  for  visual  servoing,  in 
terms  of  well-defined  control  theory,  has  not  appeared  in  literature  except  for  a  simple 
case  [Coulon  83].  Visual  servo  controllers  have  been  designed  using  ad-hoc  strategies  [Agin 
79].  In  this  section,  the  dissertation  contributes  to  the  basic  understanding  of  visual  servo 
control  by  organizing  previously  implemented  and  proposed  visual  servo  control  strategies 
into  clearly  defined  control  structures.  It  is  shown,  in  Sections  2.4.2  and  2.4.3,  that  the  task 
reference  signal,  3je(,  and  feedback  signal,  3^  can  be  based  on  either  image  feature  space  or 
position  space  measurements.  The  robot  under  control  can  be  either  open-loop  (i.e,  without 
joint  level  feedback  •  Figure  2-1),  or  an  incremental  system  (i.e.,  with  internal  joint  level 
feedback,  and  kinematic  decoupling  -  Figure  2-6).  The  formalization  of  visual  servo  control 
structures  is  based  on  these  feedback  signal  and  robot  categorizations.  The  formalization 
clarifies  the  requirements  placed  on  the  visual  servo  controller,  and  facilitates  the  design 
process.  In  Section  2.5,  a  controller  design  for  image-based  structures  is  proposed.  The 


formalization  also  provides  a  framework  for  evaluating  different  visual  servo  control 
structures.  It  is  beyond  the  scope  of  this  dissertation  to  make  such  comparative  evaulation. 

The  characteristics  of  a  general  vision  system  are  reviewed  in  the  next  section.  The 
objective  of  the  dissertation  does  not  include  presention  or  proposal  of  a  detailed  design  for 
such  a  system. 

2.4.1 .  Computer  Vision 

A  computer  based  vision  system  is  represented  in  Figure  2-8. 
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Figu  ra  2-8:  Computer  Based  Vision  System 

Vision  transducers  typically  include  vidicons,  charge  coupled  devices  (CCD),  photosensitive 
diode  arrays,  and  structured  fiber  optic  sensors  [Agrawal  83].  The  transducer  produces  a 
2-dimensional  array,  A,  of  light  intensities.  The  array  can  be  distorted  from  an  ideal  image,  A, 
due  to: 

•  Transducer  resolution; 

•  Transducer  nonlinearities  such  as  optical  aberrations  and  out  of  being  focus; 

•  Background  lighting  conditions;  and 

•  Electronic  noise  and  drift. 

The  transduction  process  can  have  time-dependent  dynamics  such  as  the  persistence  effects 
associated  with  CCD  and  vidicon  devices.  At  slower  sampling  rates  these  effects  can  be 
neglected  [Coulon  83]. 


The  image  is  preprocessed  by  filtering  operations  to  enhance  image  quality.  The  enhanced 
image  is  then  segmented  into  connected  regions.  A  one  dimensional  feature  vector,  f,  or  a 
relational  feature  description,  which  describe  segmented  regions  of  the  image  are  then 
extracted.  Features  include  areas,  angles,  center-of-gravity,  and  intensities.  Feature 


extractors  are  typically  digital  computer  algorithms.  High  speed  optical  processors  can  also 
be  used  for  preprocessing,  segmentation,  and  feature  extraction  [Merkle  83,  Stermer  83]. 

A 

Feature  space  representations  must  be  interpreted  (signified  by  I  in  Figure  2-8)  to 

A 

recognize  both  the  object  and  its  pose  Xret(*<-nd).  Interpretation  is  based  on  3-dimensional 
models  of  the  object  and  transduction  process,  and  can  include  statistical  modeling 
considerations.  Transduction  noise  and  modeling  inaccuracies  can  degrade  the  estimate. 
The  interpretation  process  can  be  facilitated  by  operating  within  structured  lighting 
environments  [Agin  79].  The  measurement  delay,  ndT,  which  is  the  total  time  required  to 
acquire,  process,  extract,  and  interpret  the  image,  can  vary  over  wide  ranges  depending  on 
the  complexity  of  the  image.  High-speed  feature  extractors  (33-100  msec)  for  simple 
structured  images  have  been  built  or  proposed  [Bracho  82,  Vuylsteke  81]  and  systems  which 
can  operate  on  more  complex  scenes  can  be  envisioned  using  optical  processors  [Merkle  83]. 
Additionally,  special  purpose  optical  proximity  sensors  have  been  constructed  which  can 
measure  positions  and  orientations  of  a  plane  in  about  1  msec  [Kanade  83]. 

Visual  control  structures,  based  on  both  image  feature  space  and  position  space 
measurements,  will  now  be  described.  Control  structures  can  be  categorized  as  either 
position-based  or  image-based.  This  formalization  was  first  introduced  by  Sanderson  and 
Weiss  [Sanderson  82],  While  individual  structures  are  described  as  autonomous  processes, 
the  possibility  of  a  complete  system  implementation  which  uses  combinations  and 
augmentations  of  these  structures  is  a  viable  concept. 

2.4.2.  Position  Based  Visual  Control  Structures 

Position-based  control  structures  define  task  reference  signals  in  world  space  position 
coordinates.  The  image  interpretation  step  provides  feedback  by  producing  position 
estimates  whose  expectations  are  generally  linearly  related  to  actual  positions. 

2. 4. 2.1.  Static  Position  Based  "Look  and  Move" 

The  static  position  based  "look  and  move”  structure  (Figure  2-9)  is  used  most  often  in 
present  industrial  applications  [Tani  77,  Birk  79,  Kashioka  77].  In  the  "look"  stage,  the  vision 
system  estimates  the  object  pose.  The  control  computer  calculates  the  error,  AX,  between  a 
reference  and  the  measured  pose.  A  "move"  by  the  incremental  error  command  is  then 
issued  to  an  incremental  robot  system.  In  this  structure,  the  incremental  robot  system  only 
accepts  new  commands  after  it  has  completed  the  previous  commanded  motion.  This 


characteristic  is  typical  of  commercially  available  robots  where  the  programmer  can  only 
access  higher  level  control  functions.  The  control  operation  is  sequential  and  asynchronous, 
and  the  sequence  is  repeated  until  AX  is  within  specified  tolerance  bounds. 


("and  move") 


noise 

Figure  2-9:  Static  Position -Based  "Look  &  Move" 


When  the  camera  is  attached  to  the  robot  end-effector  (according  to  the  transformation 
T  ).  then  Xre(  becomes  the  relative  position  between  the  camera  and  object.  In  this  case  the 
tool  transform  is  assigned  according  to: 

"^"tool*-  "^cam 

When  the  camera  is  stationary,  T(o<j|  can  be  either  the  position  of  the  end-effector  tool 
mounting  or  the  object  held  by  the  manipulator. 

The  static  "look  and  move"  structure  is  not  a  dynamic  closed-loop  control  system  because 
each  operation  is  performed  sequentially  and  independently.  Thus,  the  dynamics  of  each 
operation  does  not  affect  the  overall  system.  No  additional  control  law,  except  that  used  by 
the  robot,  is  required.  The  resulting  control  system  is  slow,  unable  to  track  moving  objects, 
and  may  require  many  iterations  to  achieve  desired  accuracy.  Implementations  of  these 
structures  have  demonstrated  the  basic  concept  of  interactive  sensing. 
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2. 4. 2. 2.  Dynamic  Position-Based  "Look  and  Move" 

In  the  static  "look  and  move"  structure,  the  reference  signal  to  the  robot  can  be  updated 
only  after  the  robot  has  reached  a  previously  commanded  goal.  When  the  robot  system  is 
designed  so  that  command  inputs  can  be  synchronously  updated,  independent  of  the  robot’s 
position  and  velocity  (i.e.,  robot  can  be  moving),  then  the  "look  and  move”  structure  can  be 
configured  as  adynamic  closed-loop  sampled  data  system  (Figure  2-10 ). 
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Figure  2-10:  Dynamic  Position-Based  "Look  &  Move” 

The  closed-loop  visual  servo  system  requires  a  feedback  controller,  Cv  to  compensate  for 
system  dynamics,  with  the  added  complexity  presented  by  feedback  delays  and  measurement 
noise.  For  applications  requiring  tracking  of  moving  objects,  prediction  algorithms  have  been 
studied  [Hunt  82]  to  improve  system  steady-state  performance.  When  the  vision  system  is 
viewed  as  a  single  process,  then  nd  =  1  (where  nd  is  the  number  of  discrete  feedback  delays) 
and  the  sampling  period  is  chosen  to  be  the  net  time  required  by  the  individual  vision 
processes.  Alternatively,  shorter  intervals  can  be  chosen  by  considering  the  vision  system  as 
a  pipe  line  of  the  individual  processes.  In  this  case,  nd>1  and  the  time  interval  is  determined 
by  the  maximum  individual  vision  component  processing  time.  From  a  control  viewpoint, 
systems  become  more  difficult  to  control  with  added  delays,  even  if  the  sampling  period  is 
reduced.  Sampling  periods  of  100  ms  have  been  suggested  as  being  sufficient  for  visual  servo 
control:  shorter  sampling  periods  would  be  required  for  high-speed  control. 

Agin  [Agin  79]  was  the  first  to  implement  dynamic  visual  control  of  a  multi-DOF  manipulator. 
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The  camera  was  mounted  to  the  end-effector  and  interpretation  was  facilitated  by  the  use  of 
structured  light-striping  techniques.  Simple  proportional  controllers,  sampling  at  between 
150  to  500  msec,  were  manually  tuned  to  achieve  the  desired  response.  Control  system 
analysis  did  not  accompany  these  experinints.  Geschke  [Geschke  82],  who  has  implemented 
similar  multi-DOF  visual  servoing  experiments,  at  100  ms  sampling  periods,  noted  that  the 
independent  closed-loop  joint-level  robot  servo  controllers  required  high  damping  at  these 
sampling  periods,  supposedly  to  avoid  exciting  structural  resonances.  Other  examples  of 
dynamic  control  are  reported  in  [Albus  81]  and  [Ward  79],  To  the  best  of  the  author’s 
knowledge,  the  only  reported  research  which  uses  control  system  analysis  to  design  a 
dynamic  visual  servoing  system  is  presented  by  Coulon  [Coulon  83].  In  this  research,  a  fixed 
proportional  +  summation  controller  is  designed  for  single- DOF  linear  system. 

2. 4. 2. 3.  Position  Based  Visual  Servoing 

Since  visual  sensors  provide  direct  world  space  information,  and  thus  world  space  servo 
errors,  it  should  be  possible  to  control  the  open-loop  robot  dynamics  and  kinematics  directly 
and  eliminate  the  "arm-solution"  or  Jacobian  required  by  the  "move"  commands  to  a  closed- 
loop  robot  positioning  system.  This  configuration  is  termed  Position -Based  Visual  Servoing 
(Figure  2- 11). 


noise 

Figure  2*1 1 :  Position-Based  Visual  Servoing 

This  configuration  may  have  potential  advantages  including: 

1.  Elimination  of  added  computational  delays,  required  by  the  "arm-solution" 
evaluation  period,  which  detract  from  relative  stability; 


2.  Elimination  of  "arm-solution"  inaccuracies;  and 


3.  Increased  efficiency,  since  the  "arm-solution"  does  not  have  to  be  evaluated,  and 
independent  joint  controllers  are  not  required.  However,  the  sampling  period 
dictated  by  the  vision  system  may  be  too  long  to  control  the  robot,  depending 
upon  the  robot’s  dynamic  and  mechanical  properties.  In  these  cases, 
independent  joint-level  velocity  feedback  controllers,  which  operate  at  higher 
sampling  rates,  can  be  used.  These  cases  are  discussed  in  Chapters  3  and  8. 

In  addition  to  measurement  delays  and  noise,  the  visual  feedback  controller,  C2,  must 
compensate  for  possibly  nonlinear  and  coupled  robot  dynamics  and  kinematics. 


Implementions  or  simulations  of  this  structure  have  not  appeared  in  literature,  in 
relationship  to  visual  servo  control.  While  Koivo  [Koivo  81]  and  Takeyaki  [Takegaki  81]  did 
not  mention  visual  servoing  ,  they  did  propose  adaptive  feedback  controllers  for  combined 
dynamic  and  kinematic  control.  They  assumed  that  the  world  coordinate  position  of  the  end- 
effector,  as  predicted  by  a  direct  kinematic  solution,  could  be  used  as  the  feedback  signals, 
but  they  did  not  consider  measurement  delays.  Takegaki  reported  "effective"  control  with  this 
approach  in  limited  simulations  of  an  output-error  type  controller  using  a  Lyapunov 
adjustment  mechanism. 


2.4.3.  Imaged  Based  Visual  Control  Structures 

In  the  position-based  approaches,  the  image  interpretation  step  can  be  complex  and  affect 
control  system  design  and  performance  by 

1.  Adding  additional  time  delays,  and 

2.  Introducing  measurement  noise  resulting  from  inaccuracies  of  the  object  and 
transduction  modeling. 

As  discussed  in  the  remainder  of  this  chapter,  image  features  are  often  continuous  functions 
of  object  position,  and  for  fixed  ranges  of  control,  could  provide  sufficient  information  for 
robot  control  without  carrying  out  the  interpretation  process.  Thus,  the  continuity  of  feature 
representations  of  time-varying  imagery  can  lead  to  potential  simplifications  of  the  control 
strategy  including  task  training  by  a  "teach-by-showing"  operation. 
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2.4.3. 1 .  Dynamic  Image-Based  "Look  and  Move" 

The  feedback  path  of  the  dynamic  position-based  "look  and  move"  structure  (i.e,  the  vision 
measurement  system  in  Figure  2-8)  can  be  decomposed  into  two  nonlinear  transformations. 
The  transduction  and  feature  extraction  functions,  or  world-to-feature  space  transformation, 
can  be  viewed  as  the  inverse  of  an  ideal  interpretation,  in  the  absence  of  noise,  according  to: 

where  I  is  "ideal"  in  the  sense  of  being  based  on  exact  models.  The  feedback  path  is  then 
mapped  to  world  space  by  the  interpretation  transformation: 

A  A 

If  the  ideal  interpretation  has  a  unique  inverse  mapping,  over  the  control  region  of  interest, 
such  that  X.  ,  are  single-valued  functions  of  f,  then  this  suggests  that  the  system  can  be 
controlled,  to  unique  end-points,  using  f(k)  directly  as  the  feedback  and  reference  signals, 
thus  eliminating  the  interpretation  step.  This  condition  must  also  be  satisfied  for  achieving 
unique  solutions  with  the  position- based  approaches.  The  uniqueness  condition  is  satisfied, 
for  the  control  region  of  interest,  when  [Wylie  61]: 

1 .  The  first  partial  derivatives  of  f  are  continuous,  and 

2.  If  the  Jacobian  of  the  ideal  inverse  interpretation  is  nonsingular;  i.e., 


det  =  det[Jfegt]  *0  (2.6) 

°-rel 

where  Jfeat  is  defined  as  the  feature  sensitivity  matrix. 

In  practice,  Jfea,  could  be  measured  on-line  to  test  the  condition  in  (2.6).  This  condition  must 
be  true  for  both  position  and  image-based  approaches.  Since  the  dissertation  does  not 
directly  address  the  imaging  problem,  features  which  are  unique  will  be  chosen.  Further, 
since  the  determinant  is  only  defined  for  square  matrices,  then  the  permitted  number  of 
degrees-of-freedom  must  equal  the  number  of  measured  features. 


The  dynamic  image-based  "look  and  move"  structure,  shown  in  Figure  2-12,  which  uses 
feature  feedback,  was  first  proposed  by  Sanderson  and  Weiss  [Sanderson  82].  In  addition  to 
compensating  for  robot  dynamics,  measurement  noise,  and  feedback  delays,  the  feedback 
controller,  C3  must  also  compensate  for  the  nonlinear  and  coupled  feature  transformations. 

Reference  signals,  Ire,(k),  must  now  be  defined  in  feature-space.  The  task  could  first  be 
defined  in  world  space,  X d  t.  and  then  mapped  into  f  (  according  to  an  idealized  inverse 
interpretation.  Equivalently,  if  JfeatQlre,)  is  known,  or  can  be  measured,  and  the  initial 


Figure  2-12:  Dynamic  Image-Based  "Look  &  Move" 

displacement  X.r°e|  is  known,  then  the  feature  signal  can  be  derived  by  evaluating  the  feature 
sensitivity  matrix  along  (  +  SX.rej  fe(  according  to 
5i  =  Jfeat5*re. 

Both  approaches  still  require  an  interpretation  step,  but  may  have  potential  advantages  by 
eliminating  inaccuracies  of  the  actual  interpretation,  in  the  feedback  path,  and  by  requiring 
smaller  sampling  periods  as  a  result  of  the  elimination  of  the  feedback  interpretation  delay. 
Alternatively  the  ref«ience  signal  could  be  defined  directly  in  image  feature  space  using  a 
"teach-by  showing"  strategy  whereby  an  image  is  transduced  in  the  uesired  reference 
position  and  the  corresponding  extracted  features  represent  the  reference  features.  When  the 
task  is  repetitive  and  the  desired  path  or  trajectory  remains  constant,  the  reference  feature 
signals  can  be  defined  a’priori  as  a  "moving"  or  time-varying  image  along  the  path.  If  the 
object  is  allowed  to  be  in  random  poses,  then  only  the  final  feature  values  can  be  defined 
a'priori.  While  the  path  cannot  be  directly  specified,  absolute  positioning  can  still  be 
achieved.  The  most  useful  applications  of  image-based  systems  might  be  for  tasks  requiring 
fast  and  accurate  corrective  motions,  where  exact  path  is  not  critical  (e  g.,  for  precision 
assembly  including  random  part  aquisition  and  parts  alignment). 

While  path  cannot  be  directly  controlled  with  the  IBVS  "teach-by-showing"  strategy, 
interesting  relationships  between  predicted  pathes  (i.e.,  the  path  that  would  result  if  the 
control  system  achieved  perfect  tracking  of  the  feature  reference  signals)  and  feature  control 


are: 


1 .  If  the  coupled  feature  sensitivity  matrix  Jf  is  constant,  and  each  feature  is 
specified  to  have  identical  time  responses,  the  predicted  path  is  straight  line 
(Section  2.5.2. 2). 

2.  For  the  2  DOF  system  evaluated  in  Chapter  6,  for  which  J(ea(  is  coupled  and  varys 
dramatically  over  the  trajectories,  the  predicted  pathes  are  either  straight-line,  or 
approach  straight- lines  (Section  6.2. 1.1). 

3.  For  the  3  DOF  system  evaluated  in  Chapter  7,  for  which  Jf  is  also  coupled  and 
varys,  the  observed  pathes  are  smooth  (i.e.,  not  tortuous),  and  approach  straight¬ 
lines  as  features  are  selected  which  reduce  system  coupling.  (The  index  of 
system  coupling  is  described  in  Section  2.5.2.) 

Such  results  are  encouraging  for  future  investigation  of  more  general  systems,  and  suggest 
that  features  may  be  found  which  can  be  used  to  achieve  some  degree  of  path  control.  As  an 
alternative  to  path  control,  a  path  constraint  strategy  is  proposed  in  Section  2.5,  and 
evaluated  in  Chapters  6  and  7. 

The  "teach  by-showing"  approach  presents  additional  requirements  for  controller  design. 
This  approach  assumes  that  the  inverse  transformation  I'1  is  unknown.  Therefore,  the 
controller  C3  must  be  based  on  a  design  approach  which  not  only  compensates  for  the 
nonlinear  and  coupled  properties  of  I'1,  but  also  for  unknown  values.  These  control 
requirements  are  discussed  in  detail  in  Section  2.5,  along  with  the  issue  of  selection  process. 
That  is,  how  should  a  subset  of  n  features  be  selected  for  control  froin  a  set  of  m  extracted 
image  features  f(  (i  =  1  ....n),  where  m>n? 

In  any  of  the  visual  based  control  structures,  some  a’ priori  knowledge  of  object  geometry 
may  be  required  for  feature  identification  and  a  level  of  interpretation  may  be  required  for 
object  recognition.  In  many  industrial  vision  applications,  identification  and  recognition  is 
based  on  the  "teach-by-showing"  principle  using  statistical  feature  information  generated  by 
image  training  sets.  Other  approaches  may  only  require  qualitative  and  relational  information 
for  identification  and  recognition.  Thus,  detailed  a'priori  modeling  would  not  bo  required  at 
this  level  of  vision  processing. 

2. 4. 3. 2.  Image-Based  Visual  Servoing 

An  image-based  version  of  the  position-based  visual  servoing  structure  can  also  be 
formulated  by  controlling  the  open  loop  robot  directly  and  eliminating  the 
"arm-solution"  [Sanderson  83a] .  This  approach  is  called  image-based  visual  servoing  (IBVS) 
(Figure  2-13).  The  physical  variable  under  control,  g(k),  can  be  related  to  f (k)  by 
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Figure  2-13:  Image- Based  Visual  Servoing 
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The  feedback  path  is  characterized  by  an  overall  small-signal  sensitivity  matrix  J  given  by 


J=Jfeat  Jami 


where 


The  IBVS  approach  is  similar  to  the  dynamic  "took  and  move"  structure  except  that  the 
design  of  the  feedback  controller,  C4,  is  further  complicated  by  having  to  compensate  for  any 
nonlinear  and  coupled  robot  dynamics  and  kinematics.  In  the  "look  and  move"  structures,  the 
robot  system  is  ideally  linear  and  uncoupled,  thus  simplifying  control  system  design. 

The  image-based  structures  are  representative  of  sensor  based  control  structures  which 
include: 

1.  A  measurement  system  which  generates  sensory  information  that  is  difficult  to 
interpret; 

2.  A  sensory  feedback  path  which  can  be  nonlinear,  coupled,  noisy,  contain 
unknown  parameters,  and  include  delays;  and 

3.  A  robot  under  control  which  can  be  dynamically  and  kinematically  nonlinear  and 
coupled. 

A  control  approach  for  image-based  systems  may  then  have  potential  for  application  to  other 


sensory  modalities  (e.g.,  touch  and  acoustic).  Because  of  this  potential  and  the  novelty  of  this 
approach,  the  remainder  of  the  dissertation  is  concerned  with  a  detailed  analysis,  design  and 
evaluation  of  "teach-by-showing"  image-based  systems.  The  "teach-by  showing"  approach 
is  analogous  to  the  basic  robot  control  "teach-mode"  operation.  In  the  author’s  experience, 
the  "teach-mode"  strategy  for  robot  task  definition  has  greatly  facilitated  the  incorporation  of 
robots  into  the  industrial  environment  by  providing  a  simple  means  for  task  training.  Similarly, 
the  "teach-by  showing"  approach  has  the  potential  of  simplifying  applications  of  visual 
servoing. 

2.5.  Control  Of  Image-Based  Systems 

A  control  law  approach  for  image  based  control  structures  is  presented  in  this  section. 
Several  characteristics  of  these  structures,  which  are  noted  in  Section  2.4,  will  present 
particular  challenges  for  design  of  a  suitable  controller.  These  include: 

1 .  Nonlinearities:  dynamics  (including  actuator),  kinematics,  and  feature-space 
transformations; 

2.  Unknown  parameters:  limited  a- priori  knowledge  (e.g.,  in  "teach-by-showing"), 
parameter  variation  and  uncertainty: 

3.  Measurement  delay; 

4.  Measurement  noise; 

5.  Feature  selection  and  assignment  (the  feature  assignment  process  addresses 
issues  in  uncoupled  control  of  coupled  plants); 

6.  Path  control:  when  tasks  are  specified  directly  in  feature  space;  and 

7.  Tasks  requiring  tracking  of  moving  objects. 

The  final  aspect  implies  that,  in  designing  the  control  system,  a  distinction  is  made  between 
visual  servoing  applications  where  the  object  and  camera  are  stationary  versus  moving  in  the 
steady-state.  An  example  is  the  case  of  an  object  moving  with  velocity  V  which  is  tracked  a  by 
a  system  which  does  not  include  a  prediction  algorithm  and  has  a  unit  feedback  delay  of  T 
seconds.  If  the  steady-state  error  between  the  reference  and  delayed  feedback  signals  can 
be  driven  to  zero,  then  an  actual  tracking  error  of  VT  remains.  Steady-state  servo  errors  can 
be  made  small  with  a  sufficiently  large  bandwidth  controller,  but  controller  design  trade-offs 
are  usually  made  between  transient  response  and  steady-state  errors.  As  T  increases  (due  to 
vision  processing  constraints),  the  controller  bandwidth  is  forced  to  be  smaller.  The  steady- 


state  servo  error  and  actual  tracking  error  increase,  and  a  prediction  algorithm  is  required  to 
reduce  tracking  errors.  For  tasks  involving  stationary  objects,  a  controller  can  always  be 
designed  to  reduce  steady-state  position  errors  to  zero,  irrespective  of  controller  bandwidth 
or  the  use  of  prediction  algorithms.  For  these  applications,  transient  performance  becomes 
the  important  design  goal.  As  an  initial  step  in  understanding  image-based  control,  prediction 
algorithms  are  not  included  in  this  dissertation  and  only  tasks  involving  stationary  steady- 
states  are  considered.  Step  input  reference  signals,  defined  by  the  "teach-by-showing" 
strategy,  provide  a  suitable  measure  of  system  performance. 

In  this  section,  a  control  law  approach  is  introduced,  and  the  controller  is  designed  in 
Chapter  3.  The  feature  selection  and  assignment  issues  are  addressed  by  defining  a  measure 
of  feature  transformation  coupling.  And,  finally,  methods  for  path  constraint  are  suggested. 

2.5.1 .  Control  Approach 

To  design  a  controller,  it  is  useful  to  consider  small-signal  models  of  the  system.  Small- 
signal  models  (about  a  nominal  operating  point  or  trajectory)  of  "look  and  move"  and  IBVS 
structures  are  represented  in  Figures  2-14  and  2-15,  respectively. 
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noise 


Figure  2-15:  Small-Signal  Image- Based  Visual  Servoing 
5Xre|  =  -SX, 

for  a  stationary  object  or  camera.  The  system  output  is  the  undelayed  feature,  while  the 
feedback  path  is  modeled  by  discrete  unit  delays.  The  incremental  robot  system  is  W^z’1) 
(Section  2.3.2. 1).  Linearized  open-loop  robot  dynamics  [Chung  82],  or  equivalent  linear  I/O 
modeles  [Stone  84],  are  represented  by  W^z'1).  In  addition  to  the  control  requirements  of  the 
robot  dynamics,  described  in  Sections  2.3  and  2.4,  the  design  of  the  controller,  C3  or  C4,  also 
depends  on  the  "J"  sensitivity  matricies,  feedback  delays,  and  measurement  noise.  In 
Chapters  5,  6,  and  7,  the  sensitivity  matricies  are  shown  to  be  nonlinear  and  coupled 
functions  of  g  and  X  ;  thus,  J  varies  as  g  varies,  and  feature-space  transformations  are 
manifested  by  time-varying  open-loop  gains.  Predicted  values  of  J  can  deviate  from  actual 
values  due  to  inaccuracies  in  the  modeling  of  the  three  dimensional  object  and  transduction 
process,  and  from  drift  and  variation  in  the  transducer  parameters.  At  the  extreme,  the  values 
may  be  completely  unknown  a'priori  when  minimal  knowledge  of  the  inverse  interpretation 
transformations  T1  are  available,  such  as  arises  when  task  programing  is  limited  to  the 
"teach  by-showing"  strategy. 

Fixed  feedback  controller  designs  have  limitations  in  the  control  of  such  nonlinear  and 
unknown  systems.  Even  if  the  nonlinearities  are  known,  a  fixed  controller  design  for  these 
systems  is  a  formidable  engineering  problem.  In  contrast,  the  adaptive  approach  to  controller 
design  appears  to  be  applicable  for  these  requirements.  The  adaptive  approach  has  already 
been  suggested  and  studied  in  application  to  nonlinear  dynamic  and  kinematic  control  of 
possibly  unknown  plants  (Section  2.3.1. 3).  It  would  appear  that  this  approach  is  also  capable 
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of  adapting  to  nonlinear  and  unknown  feature  gains  (Jfeat  or  J).  The  IBVS  controller  design 
developed  in  the  dissertation  will  therefore  emphasize  the  adaptive  approach.  Performance 
limitations  and  application  of  fixed  controllers  are  also  evaluated  by  fixing  the  adjustable 
gains  of  the  adaptive  controller  (derived  from  initial  adaptive  learning  trials)  in  the  simulation 
experiments. 

Adaptive  and  fixed  controllers  can  be  coupled  or  uncoupled.  Coupled  controllers  have  an 
inherently  greater  potential  for  being  able  to  uncouple  a  coupled  system.  Uncoupling  of  a 
feature  based  input/output  system,  however,  does  not,  in  general,  uncouple  the  world  space 
variables.  Coupled  controllers  have  several  potential  disadvantages,  including: 

1. They  are  computationally  complex  relative  to  uncoupled  controllers.  For 
example,  it  is  shown  in  Chapter  8.  that  the  computational  copmplexity  of  an  m- 
input,  m-output  nth  order  MIMO  MRAC  controller  is  order  m^n2,  versus  inn2  for  m 
independent  SISO  controllers; 

2.  MIMO  controllers  do  not  lend  themselves  to  modularity.  A  modular  system  can 
easily  be  extended  to  increasing  degrees-of-freedom,  and  distributed  processing; 
and 

3.  MIMO  feedback  controllers  make  available  extra  degrees-of  freedom  in  the 
controller,  such  that  non-unique  combinations  of  controller  gains  can  specify 
desired  pole  iocations.  Added  controller  degrees  of- freedom  essentially 
complicate  the  manual  tuning  process,  and  parameter  initializations. 

Uncoupled  adaptive  controllers  have  already  demonstrated  the  potential  to  control 
dynamically  coupled  robots.  They  are  modular,  and  computationally  simpler,  and  would  thus 
be  easier  to  implement  (in  future  research)  in  current  laboratory  and  factory  computing 
environments.  For  these  reasons,  the  approach  developed  in  this  dissertation  will  emphasize 
uncoupled  control  of  coupled  systems,  using  the  concept  of  equivalent  SISO  plants  (See 
section  2.3.1. 3).  For  example,  a  2  DOF  IBVS  system  is  controlled  by  independent  MRAC 
controllers  in  Figure  2-1G.  The  small-signal  sensitivity  matrix,  J  in  (2.7),  is  included  in  the 
figure  to  clarify  its  role  in  system  coupling.  From  the  adaptive  controller  (U,  and  J-2) 
viewpoints,  the  system  is  equivalent  to  controlling  two  independent  plants,  depicted  in  Figure 

A  A  , 

2-17.  In  the  figure,  ^  eq  (z  )  and  ei|  (z  )  are  equivalent  SISO  linear  prediction  models  with 
parameters  that  may  be  time-varying.  That  is,  W  fz  )  predicts  f  (k),  from  past  and  present 

I-6Q  I 

values  of  u((k)  and  f((k),  independent  ;f  actual  physical  plant  models. 

A  coupled  controller,  which  estimates  coupled  models,  is  outlined  in  Chapter  8  for 
computational  complexity  analysis  and  possible  future  implementation.  Problem  areas  of 
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Figure  2-16:  MRAC  Control  of  an  IBVS  System 
SISO  control,  which  are  isolated  in  Chapters  6  and  7,  suggest  the  potential  performance 
advantages  gained  by  MIMO  control. 


The  approach  to  adaptive  control  will  follow  the  enhanced  identification  error  MRAC 
developed  by  Morris  and  Neuman  [Neuman  80].  While  similar  approaches  have  appeared  in 
literature,  their  research  focused  on  details  of  physical  implementation  including  control 
signal  saturation  and  controller  stability,  measurement  noise,  and  computational  complexity 
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Figure  2-17:  Equivalent  IBVS  Control 

for  microprocessor  implementation.  Since  the  algorithm  does  not  include  control  of  systems 
with  discrete  measurement  delays,  the  algorithm  is  extended  in  Chapter  3  to  include  control  of 
systems  with  delay.  Additional  modifications  for  applying  uncoupled  MRAC  to  the  control  of 
coupled  nonlinear  systems  are  also  developed. 

2.5.2.  Feature  Selection  and  Assignment 

Feature  transformation  coupling  (i.e.,  represented  by  the  small-signal  feature  sensitivity 
matrix  J)  leads  to  related  problems  of  feature  selection  and  assignment.  The  feature  selection 
process  asks  the  question:  How  should  a  subset  of  n  features  be  selected  from  a  set  of  m 

possible  control  features  f  (i  =  1 . ,m),  where  m>n?  And,  the  feature  assignment  process 

asks:  Since  the  uncoupled  controllers  are  used  to  control  coupled  plants,  which  feature 
should  be  used  to  control  each  actuator?  Both  issues  are  related  to  the  degree-of-coupling  of 
the  feature  transformation.  A  measurement  of  coupling  is  proposed  to  help  answer  these 
questions. 

2. 5. 2.1.  Feature  Assignment  Using  Diagonal  Dominance 

In  applications,  where  uncoupled  controllers  are  used  to  control  coupled  plants,  there  is 
always  the  problem  of  choosing  which  servo  error  will  control  which  actuator;  that  is,  for  a  set 
of  n  outputs  yj  (i  =  1,...,n),  which  servo  error,  Ayjt  should  be  filtered  and  coupled  to  the  j,h 


actuator  as  u(?  The  control  engineer  tackles  this  problem  with  insight  into  the  physical  nature 
of  the  system  and  formulates  qualitative  notions  of  the  degree-of-coupling  between  the 


independent  actuators  and  the  system  output.  For  joint-level  robot  control,  Aqj  should  be 
used  to  derive  ur  For  other  systems,  such  as  IBVS  control,  the  assignment  may  not  be 
straight  forward.  To  formalize  this  procedure  let  the  open-loop  linear  system  be  defined  by 
Y(s)  =  H(s)u(s) 

where  H(s)  is  an  (n  x  n)  transfer  function  matrix.  When  the  system  is  uncoupled,  H(s)  can  be 
transformed  into  a  diagonal  matrix  by  switching  the  j,h  and  kth  columns  of  H(s),  and  therefore 
the  j,h  and  k,h  rows  of  u(s),  until  all  off  diagonal  elements  of  H(s)  are  zero.  When  H(s)  is 
diagonal,  the  only  choice  for  servo  error/actuator  assignment  is  a*—  Ay..  When  the  system  is 
coupled,  then  H(s)  cannot  be  transformed  into  a  diagonal  matrix.  Servo  error/actuator 
assignment  selection  can  be  accomplished  by  organizing  H(s)  in  a  'diagonally  dominant" 
fashion  [Rosenbrock  74],  such  that  the  diagonal  elements  dominate  the  off-diagonal 
elements.  Diagonal  dominance  is  defined  as 

n 

IHi;(s)l>yi  IH..(s)l  for  i  =  1  ,...n;  and  all  s  on  a  desired  contour.  (2.8) 

i  =  i 

i*i 

When  H(s)  can  be  organized  according  to  this  definition  of  dominance,  then  limited  stability 
properties  of  both  coupled  and  uncoupled  fixed  control  of  the  system  can  be 
formulated  [Rosenbrock  74].  When  applied  to  image-based  systems,  with  IJWpl<-H(s),  it  will 
be  shown  in  Chapter  6,  that  for  limited  degrees-offreedom,  the  sensitivity  matrices  cannot  in 
general  satisfy  this  definition  of  dominance. 


An  alternative  approach  is  to  organize  JWp  to  maximize  the  inequality  (2.8)  over  all  possible 
column  arrangements.  This  criterion  reduces  to  defining  the  dimensionless  measure  of 


diagonal  dominance  as 

n  n 

D(k)  =  log^ 

i  =  i  j=i 

j*i 


UWn-iiW' 

I  JWp..(k)l 


(dimensionless) 


and  then  minimizing  D(JWp)  over  all  n!  possible  column  arrangements.  The  logarithm  of  the 
dominance  is  used  since  the  ratios  change  by  orders  of  magnitude.  The  suitability  of 
minimizing  D(JWp)  to  predict  a  suitable  feature/joint  assignment2  will  be  evaluated  in 
Chapters  6  and  7.  In  these  evaluations,  it  is  assumed  that  the  open  loop  plant  time  constants 


*The  nomenclature  "feature/joint  assignment",  throughout  the  dissetation,  is  used  to  mean  the  feature  to  actuator 
servo  error  assignment  issue. 


are  much  smaller  than  the  closed  loop  system  time  constants.  The  plant  Wp  will  be 
approximated  by  its  low  frequency  gains  for  calculation  of  dominance.  This  corresponds  to 
the  DC  gains  for  Type  0  plants.  For  Type  I  plants,  the  low-frequency  gain  of  the  open-loop 
integrator  is  used. 

Since  J,  and  possibly  Wp,  are  time-varying,  the  assignment  might  require  on-line 
reassignment  during  control.  This  would  require  J  and  Wp  to  be  known  a'priori  or  estimated 
on-line.  In  the  2  DOF  evaluations  (in  Chapter  6),  J  and  Wp  are  derived  analytically  for 
specification  of  feature  assignment  over  the  trajectory.  In  the  3  DOF  evaluations  (in  Chapter 
7),  IJWpl  it  is  measured  at  the  start  of  the  control  task  by  sequentially  moving  each  degree-of- 
freedom  by  small  increments,  and  measuring  the  change  in  features.  The  resulting  feature 
joint  assignment  is  then  fixed  over  the  entire  control  trajectory.  Evaluation  of  limited  degree- 
of-freedom  systems  suggests,  for  a  set  of  features,  the  feature/joint  assignment  remains 
constant  over  large  regions  of  control  for  uncoupled  kinematic  configurations  (e.g.,  Cartesian 
robot  and  "look  and  move"  structures).  For  coupled  kinematic  configurations  (e.g.,  an 
articulated  arm),  a  feature/joint  reassignment  is  predicted  when  controlling  over  large 
distances  in  space.  Thus,  measurement  of  JWp  at  the  start  of  the  control  task  may  only  be 
suitable  for  feature/joint  assignment  applications  requiring  small  corrective  motions.  A 
general  image  based  system  would  require  on-line  estimation  of  JWp  to  determine  suitable 
assignments.  A  method  for  accomplishing  this  task  is  proposed  in  Chapter  8. 

2. 5. 2. 2.  Feature  Selection 

The  image  of  a  typical  scene  generally  contains  more  features  than  there  are  degrees-of- 
freedom  to  control.  The  number  of  features  must  equal  the  number  of  degrees- of- freedom  in 
an  image  based  system  since  the  feature  sensitivity  matrix  is  constrained  to  be  square.  The 
feature  selection  question  is  then:  How  should  a  subset  of  n  features  be  selected,  for  an 
n-DOF  robot,  from  a  set  of  m  features,  where  m>n?  The  possible  number  of  candidate 
subsets  is 

Qm! 
n!(m-n)! 

If  a  coupled  control  law  is  used,  then  the  ordering  of  the  feature  subset  is  not  an  issue.  The 
process  of  feature  selection  becomes  more  complex  when  an  independent  control  law 
strategy  is  used.  The  feature/joint  assignment  issue  increases  the  total  number  of  ordered 
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How  can  a  subset  be  chosen  from  p(m,n)  potential  candidates?  To  arrive  at  a  strategy,  it  is 
useful  to  first  describe  the  idealized  attributes  for  a  feature  subset.  While  such  attributes  may 
not  be  expected  in  practice,  they  can  serve  as  guidelines. 


To  arrive  at  a  criterion  for  feature  selection,  two  aspects  of  feature-based  control  are 
analyzed: 

1 .  Abilty  to  specify  world  space  path  using  feature  based  trajectories  (assuming  that 
the  control  system  can  achieve  a  specified  feature  space  performance),  and 

2.  The  control  effort  required  to  achieve  the  specified  feature  space  dynamic 
performance. 

It  is  shown  below  that  the  attributes  of  the  feature  sensitivity  matrix,  Jfeat,  relate  to  path 
performance,  while  the  attributes  of  JWp  relate  to  the  control  effort  aspects. 


With  respect  to  world  space  path,  it  is  desireable  to  be  able  to  control  independently  each 
world  level  DOF.  To  achieve  this  goal,  an  ideal  subset  of  features  should  yield  a  feature 
sensitivity  Jfeat  which  is  diagonal  and  constant.  Then, 

AX.=  ~~i- 

»  I 

reat  i ,  i 

where,  AX.  is  the  path  error  for  the  i,h  DOF,  Af.  is  the  i,h  feature  error,  and  J,  ,  .  is  the  (i,i),h 
element  of  Jfeat.  If  straight-line  motion  is  desireable,  and  all  of  the  features  exhibit  the  same 
dynamic  response,  then  straight-line  motion  would  be  achieved.  For  example,  assume  that  the 
i,h  feature  response  is  specified  by  the  critically  damped  response 

fjo(t)=f.(t)-f.(0)  =  Af.(l.e,/T) 

and  all  feature  responses  have  the  same  time  constant  t.  The  response  of  the  i,h  DOF  is 
XjO(t)=Xj(t)-X(0)  =  Jjgal.j  j  f* (t) 


The  relationship  between  any  two  Cartesian  degrees-of-freedom  becomes 


X. 


_ feat-i _ i 


.  Af.  1-e 


t/T 


Af.  1-e 


-t/T 


=  Constant 


J  feat-i  — i 

which  is  constant  and  specifies  the  equation  of  a  straight-line  in  Cartesian  coordinates.  It  thus 
becomes  straightforward  to  specify  straight-line  motionl 
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If  an  ideal  feature  sensitivity  matrix  could  be  synthesized,  then  it  still  remains  to  control 
dynamically  the  system  to  achieve  the  desired  feature  response.  Attributes  of  the  overall 
sensitivity,  JWp,  can  be  used  to  describe  the  control  effort  required  to  achieve  the  desired 
response.  Similar  to  the  feature  sensitivity  attributes,  the  idealized  overall  sensitivity  matrix 
should  be  diagonal  and  constant.  Diagonalization  permits  the  unqualified  use  of  independent 
controllers.  And,  constant  elements  would  permit  the  use  of  fixed  controllers,  or  using  an 
adaptive  approach,  would  require  lower  sampling-to-bandwidth  ratios  (Section  3.6.2). 

The  evaluations,  in  Chapters  6  and  7,  demonstrate  that  the  idealized  sensitivity  attributes 
cannot  be  expected  in  practice.  The  matricies  are  coupled  (i.e.,  not  diagonal)  and  typically  are 
time-varying.  Feature  sensitivity  changes  are  minimized  for  small  motion  tasks,  and  for 
configurations  with  large  magnifications.  If  the  feature  sensitivity  were  constant,  but  coupled, 
the  predicted  path  would  still  be  straight-line  motion!  Since 


X  =J 


1 


f 


feati 


then 


=  K.(1-es/T) 


where  K.  is  a  constant.  The  constant  relationship  between  any  two  Cartesian  DOF  becomes 
X.°/X.0=  K./K. 

*  i  'I 

From  the  dynamic  control  viewpoint,  it  will  be  shown  that  SISO  independent  controllers  can 
be  used  to  control  these  feature  coupled  systems,  with  limitations.  An  MIMO  control  law  might 
produce  superior  performance,  but  with  added  computational  complexities. 


Since  we  may  not  expect  to  find  feature  subsets  which  yield  idealized  sentitivity  attributes,  a 
feature  selection  strategy  could  seek  a  subset  which  best  approximates  these  ideals;  i.e., 
select  features  which  minimize  the  coupling  and  senitivity  changes  along  a  trajectory.  The 
dissertation  will  limit  its  scope  by  focusing  on  feature  coupling  issues  for  feature  selection. 
The  question  which  again  arises  is  how  to  quantify  degree-of-coupling.  In  the  dissertation,  the 
diagonal  dominace  measure,  D(JWp),  in  (2.9),  will  be  used  to  quantify  system  coupling.  The 
feature  selection  strategy  then  becomes  minimizing  D(JWp)  and  D(J(eat)  over  the  set  of 
candidate  features.  By  minimzing  D(J  Wp),  improved  dynamic  response  is  expected  with  SISO 
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controllers.  And,  by  minimizing  D(Jfea(),  closer  to  monotonic  path  performance  would  be 
expected.  Each  strategy  may  not  produce  mutually  exclusive  decisions,  and  arbitration 
between  them  would  be  based  on  the  relative  importance  of  each  attribute.  For  example,  a 
system  could  be  feature  uncoupled  in  the  joint  space  of  an  articulated  robot  arm,  but  not 
uncoupled  in  Cartesian  space.  Since  the  degree-of-coupling  plays  such  an  important  role  in 
the  independent  control  approach,  the  dissertation  will  limit  the  evaluations  to  feature 
selection  based  on  minimization  of  D(JWp). 

Finally,  when  designing  an  image-based  "look  and  move1’  system,  the  feature  selection 
process  can  equivalently  be  viewed  as  a  world  coordinate  system  selection  process.  That  is, 
the  "arm-solution"  could  be  based  on  Cartesian,  cylindrical,  or  spherical  reference  frames. 
For  a  subset  of  features.  Jf  may  be  less  coupled  in  one  world  frame  than  in  others.  While 
this  approach  is  not  evaluated  in  this  dissertation,  this  is  an  important  topic  for  future 
research. 

2.5.3.  Path  Constraint 

In  this  section,  the  issue  of  path  constraint  is  addressed,  when  the  task  is  specified  directly 
in  feature  space  and  objects  can  be  in  random  poses  so  that  only  the  end-points  of  the 
reference  commands  are  known  a’priori.  In  this  case  the  exact  path  cannot  be  specified.  For 
example,  for  a  "look  and  move"  structure,  with  Jfeat  uncoupled  (i.e.,  D=  -oo),  a  monotonic 
path  can  be  predicted.  When  J  is  coupled,  the  path  which  produces  asymptotic  feature 
errors  need  not  be  monotonic.  When  this  occurs,  task  performance  can  be  affected  as 
follows: 

1.  An  object  image  can  be  driven  out  of  the  field -of- view,  and  the  feature  control 
must  be  stopped; 

2.  A  potential  for  collisions  with  other  objects  in  the  surrounding  environment  can 
arise; 

3.  Longer  distances  to  travel  increase  the  response  time;  and 

4.  Overshoot  can  occur,  resulting  in  physically  bumping  into  the  object. 

The  alternative  to  path  control  becomes  path  constraint.  In  this  approach,  independent 
robot  degrees-of- freedom  (i.e.,  joint  level  in  IBVS  and  world  level  in  "look  and  move")  are 
constrained  by  dynamic  braking  (by  position  servoing),  disabling  (by  mechanical  braking),  or 
by  forcing  direction  reversal  along  that  degree-of-freedom.  Constraint  is  based  on  on-line 
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measurements  of  path  performance.  For  example,  path  performance  can  be  specified  in  robot 
world  space  by  requiring  that  either  boundaries  and/or  directions  of  travel  are  constrained  to 
limits  defined  by  a'priori  task  knowledge.  Directional  constraints  may  arise  naturally  from  the 
task  requirement  (e.g.,  the  object  is  always  in  front  of  or  to  the  right  of  the  initial  robot 
position).  Work  cell  boundaries  can  also  be  specified.  During  control,  g  is  measured  and 
robot  world  coordinate  positions  are  predicted  using  the  direct  kinematic  solution.  Deviations 
from  specified  directions  or  across  boundaries  signal  the  requirement  to  constrain  motion 
along  the  responsible  degrees-of-freedom.  In  the  "look  and  move"  structures,  it  is 
straightforward  to  command  independent  world  coordinates.  In  the  IBVS  structure,  the 
problem  becomes  more  difficult  without  the  internal  compensation  for  nonlinear  kinematics, 
and  only  the  generalized  (or  joint)  degrees  of  freedom  which  "dominate"  the  specified  world 
coordinates  can  be  constrained.  Generalized  coordinate  domination  is  then  predicted  based 
upon  the  diagonal  dominance  minimization  of  J  (i.e.,  q.  dominates  X.  when  J  is 

arm  i  j  arm 

organized  to  minimize  D(Jafm).  Directional  constraints  might  also  be  evaluated  by  using 
feature  space  measurements  and  on-line  estimates  of  Jf  according  to 


5X 


— rel 


'feat 


5f 


If  a'priori  task  constraints  are  not  specified,  can  feature  measurements  alone  be  used  to 
constrain  motion,  anti  if  so  how?  While  a  genera)  approach  for  feature  based  path  constraint 
has  not  been  developed,  a  potential  strategy  has  evolved,  and  is  evaluated  in  the  dissertation. 
In  this  approach,  a  hierarchy  of  control  is  established  (Figure  2-18).  At  the  top  level  of  control, 
limited  degrees  of  freedom  are  under  feature  control,  while  the  remaining  are  constrained.  In 
proceeding  from  top  to  bottom,  additional  degrees-of-freedom  become  available  for  feature 
control.  At  the  bottom  level,  all  degrees-of-freedom  are  under  feature  control.  At  each 
sampling  instant,  decisions  are  made  to  determine  the  operating  level.  These  decisions  are 
based  on  feature  performance  indices,  PI  (defined  below),  compared  with  a  threshold  e.  By 
not  forcing  all  degrees-of-freedom  to  be  turned-on  initially,  this  strategy  may  minimize  non¬ 
monotonic  path  deviations.  It  remains  to  specify  how  to  select  the  degrees-of-freedom  which 
comprise  each  level,  and  performance  measures  which  dictate  which  level  is  operational. 

The  object  pose,  X  t.  is  defined  in  Cartesian  space  and  is  divided  into  three  movement 
modes  according  to: 

1.  position:  Xre|1  Zre| 


2.  orientation:  <p,e,^ 
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Pli - X  100% 

Af 
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This  measure  attempts  to  normalize  the  feature  error  relative  to  the  image  boundaries.  In  IBVS 
structures,  generalized  coordinates  which  dominate  movement  in  the  world  modes  are 
selected  according  to  the  diagonal  dominance  of  J  (i.e.,  minimization  of  D(J  )).  The 
resulting  hierarchy  is  represented  in  Figure  2-19. 
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Figure  2-19:  Hierarchical  Control  Example 


In  the  experimental  simulations,  joints  are  assumed  to  be  constrained  by  mechanical 
breaking,  and  the  threshold  levels  are  determined  experimentally. 
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2.6.  Summary 

The  central  theme  of  this  chapter  is  the  dynamic  control  of  robot  end-effectors.  Non-sensor 
based  robotic  systems  are  limited  to  control  systems  which  use  generalized  coordinate 
measurements  for  feedback,  while  end-effector  position  is.  then  predicted  using  kinematic 
models.  The  control  issues  involve  compensation  of  nonlinear,  coupled,  and  possibly 
unknown  robot  dynamics.  Feedforward  control  leads  to  precise  tracking,  but  is 
computationally  intensive  and  relies  on  accurate  robot  models.  Fixed  linear  feedback 
controllers  are  simple  to  implement,  but  their  dynamic  performance  can  degrade  with 
increasing  joint  speeds,  and  over  broad  ranges  of  arm  configurations.  Application  of  adaptive 
controllers  has  only  recently  be  evaluated,  but  initial  simulation  studies  suggest  that  they  are 
suitable  for  robot  control  over  a  wide  range  of  speeds  and  configurations,  and  can  be 
implemented  using  currently  available  microprocessor  technology. 

Computer  vision  extends  the  feedback  measurement  space  to  include  relative  end-effector 
position.  This  capability  permits  the  synthesis  of  control  systems  which  utilize  such 
measurements  for  control  of  end-effector  positioning.  These  integrated  visual  servo  systems 
have  the  potential  for  improving  robot  accuracy  and  extending  their  capabilities  for  operating 
in  unknown  environments  (e  g.,  objects  to  be  aquired,  or  worked  on,  are  in  random  positions). 
Control  structures  for  visual  servoing  are  formally  organized,  and  summarized  in  Figure  2-20. 
The  image-based  approaches  are  novel  structures  and  facilitate  robot  task  training  by  using  a 
"teach-by-ol towing"  strategy.  They  may  also  exhibit  superior  accuracy,  relative  to  the 
position-based  approaches,  since  they  do  not  require  explicit  interpretation,  of  image 
features,  to  measure  world  coordinates. 

For  each  control  structure  (with  the  exception  of  "static"  position-based),  a  visual  feedback 
controller  is  required  since  the  role  of  computer  vision  as  a  measurement  process  affects  the 
closed-loop  system  dynamics.  In  addition  to  robot  dynamics  and  kinematics,  the  linearity, 
noise  properties,  coupling,  unknown  parameters,  and  computational  delays  of  the 
measurement  process  must  be  considered  in  the  design  of  the  control  system.  To  meet  the 
requirements  for  image-based  control,  an  adaptive  SISO  MRAC  approach  for  control  system 
design  is  proposed.  Issues  of  feature  selection  and  assignment  are  addressed  by  defining  an 
image  feature  transformation  coupling  measure  based  on  "diagonal  dominance"  of  the 
transformation  Jacobian.  While  the  analysis  (in  this  chapter  and  Chapters  6  and  7)  shows  that 
the  predicted  position  trajectories,  for  image-based  control,  can  be  straight-lines  or  approach 
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Robot  Under  Control 


Figure  2-20:  Visual  Based  Control  Structures 
straight-lines  (for  the  systems  evaluated  in  the  dissertation),  the  end  effector  path  cannot  be 
directly  specified.  Path  control  reduces  to  a  problem  of  path  constraint.  A  hierarchical  control 
strategy,  using  feature  based  performance  measurements,  is  proposed  to  minimize  deviations 
from  a  non  monotonic  path. 


The  model- reference  adaptive  controller  is  developed  in  Chapter  3.  In  Chapter  4,  simulation 
models  are  presented  for  the  visual  servcing  tasks  to  be  evaluated  (in  Chapters  5,  6,  and  7) 
using  the  IBVS  approach. 


Chapter  3 

Model  Reference  Adaptive  Control 


3.1 .  Overview 

The  purpose  of  this  chapter  is  to  review  the  uncoupled  MRAC  control  algorithm,  and  modify 
it  for  systems  with  measurement  delay.  Additional  modifications  for  applying  uncoupled 
MRAC  to  control  of  coupled  systems  are  presented.  Interpretation  of  MRAC  control  of  a 
constant  linear  plant,  and  selection  of  MRAC  parameters  are  also  discussed.  The  controller 
described  here  is  based  on  the  algorithm  developed  at  Carnegie-Mellon  University  by  Neuman 
and  Morris  [Neuman  30,  Morris  81].  Their  research  concentrates  on  microprocessor  adaptive 
control  of  linear  SISO  processes  with  slowly  time-varying  parameters.  Similar  algorithms  have 
been  applied  in  the  literature  [Dubowsky  79,  Horowitz  80.  LeBorgne  81]  to  control  simulations 
of  nonlinear  and  coupled  robot  dynamics,  but  have  not  utilized  the  enhancements,  stability 
viewpoints,  and  attention  to  implementation  presented  in  this  dissertation. 

The  SISO  MRAC  controller,  developed  at  CMU,  is  reviewed  in  Section  3.2.  A  contribution  of 
this  dissertation  is  the  extension  (in  Section  3.3)  of  this  controller  to  control  systems  with 
measurement  delay.  In  Section  3.5,  steady-state  analysis  of  this  system  predicts  a  minimum 
requirement  for  a  control  penalty  enhancement  to  insure  relative  stabilty.  Additional 
modifications  for  applying  uncoupled  MRAC  to  control  of  coupled  systems,  including  signal 
biasing  and  gravity  compensation,  are  presented  in  Section  3.4.  Finally,  Section  3.6  includes 
guidelines  for  selection  of  MRAC  parameters. 

3.2.  MRAC  for  Systems  Without  Measurement  Delay 

The  purpose  of  this  section  is  to  review  the  SISO  MRAC  controller  developed  by  Neuman 
and  Morris  [Neuman  80].  A  detailed  derivation  and  discussion  of  this  controller  is  included  in 
Appendix  A. 
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Figure  3-1 :  Analog  Process  Under  MRAC  Control 
The  MRAC  system  comprises  three  basic  components: 

1 .  Analog  plant  under  control; 

2.  Full-parallel  reference  model;  and 

3.  Adaptive  controller 

The  reference  model  transfer  function  M(z_1)  specifies  the  desired  closed-loop  response  of 
the  sampled  plant  output,  y(k),  to  a  reference  signal  r(k).  The  adaptive  controller  is  a  digital 
feedback  controller.  The  controller  gains  are  adjusted  to  drive  the  closed-loop  response  of 
the  system  to  that  of  the  reference  model.  The  controller  uses  both  plant  and  model  input  and 
output  information  to  generate  the  control  signal  u(k). 


At  the  heart  of  the  MRAC  is  the  controller  adjustment  mechanism.  If  the  full-earallel  (FP) 
output  error  is  defined  as 

<fc)  =  x(k)-y(k),  (3.1) 

the  adjustment  mechanism  must  be  designed  so  that 

1 .  epp(k)  asymptotically  approaches  zero,  and 

2.  The  closed-loop  system  is  stable. 


Adjustable  controller  design  is  based  on  the  identification-error  [Neuman  80].  The  plant 
input-output  information  generates  estimates  of  the  plant  parameters  and  are  used  by  an 
identifier  to  predict  the  plant  output.  The  identification  error  is 

e|D(k)  =  y(k)-t{k)  (3.2) 

where  the  identifier  output  t(k)  is  the  predicted  plant  output.  This  error  drives  the  adjustment 
mechanism  which  updates  the  estimates  of  the  plant  parameters.  In  turn,  these  estimates  are 
used  to  adjust  the  controller  gains.  The  adjustment  mechanism  is  designed  so  that  e|D(k)  is 
globally  asymptotically  stable  for  a  SISO  linear  system.  The  controller  is  then  designed  to 
guarantee  the  asymptotic  stability  of  the  full-parallel  error. 

The  mathematical  structure  of  the  MRAC,  without  measurement  delay,  is  described  in 
Appendix  A. 

3.3.  MRAC  Control  With  Measurement  Delay 

The  adaptive  control  algorithm,  reviewed  in  Section  3.2,  assumes  instantaneous 
transduction  of  the  feedback  signal.  Control  systems,  such  as  IBVS,  have  time  delays  in  the 
feedback  path.  These  delays,  which  increase  the  order  of  the  system  under  control  (by  adding 
discrete  open  loop  poles  at  z  ==  1 ) ,  make  the  system  more  difficult  to  control  and  reduce  the 
margins  of  relative  stability. 

One  approach  to  the  adaptive  control  of  systems  with  delay  might  be  to  apply  the  algorithm 
reviewed  in  Section  3.2,  with  higher-order  estimators.  Preliminary  simulation  evaluation  with 
this  approach  resulted  in  unstable  systems. 

Another  approach  is  to  make  the  order  of  the  system  equal  to  that  of  the  plant  by  using  a 
predictive  algorithm  to  estimate  the  undelayed  process  output.  For  example,  Bahill  [Bahill  83] 
suggests  the  use  of  a  Smith  predictor  control  scheme  which  includes  an  adaptive  controller. 
This  approach  adds  another  level  of  complexity  to  the  control  law  design  and  evaluation.  The 
approach  followed  in  this  dissertation  is  to  use  the  delayed  output  directly  as  the  feedback 
signal  by  extending  the  basic  MRAC  algorithm  to  include  known  time  delays  in  the  control 
law.  Extension  of  the  MRAC  algorithm  to  include  measurement  delays  follows. 


figure  3-2:  MR  AC  with  Measurement  Delay 


To  modify  the  control  algorithm,  the  measurable  signal  at  t  =  kT,  yd(k),  is  assumed  to  be  the 
variable  under  control.  The  actual  process  output,  delayed  by  nd  sampling  periods,  is  yd(k).  If 
y.(k)  is  stable,  then  y(k)  must  be  stable  since  yd(k)  is  related  to  y(k)  by  the  stable  polynomial 


The  process  under  control  becomes  G(s)  in  cascade  with  nd  time  delays.  The  discrete 

process  transfer  function  Wd(z  ’)  is 

i  YJz1)  -n.  . 

W  (z' 1 )  =  — <1  — —  =  z  dW(z1) 
d  U(z  ) 
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yd(k)=£T£(k-i) 

where 

<|>d(k- 1  )=[u(k- 1  -nd)...u(k-n-nd)  yd(k-1)...yd(k-nd)]T  is  the  (2n  X  I)  information  vector 
modified  to  reflect  the  measurement  delay.  The  adjustment  mechanism  in  (A.  15),  (A.  16),  and 
(A.  17)  becomes 


A  A  1 


£(k-nd)  =  £(k-1  -nd)  +  -P(k-1  )£d(k-l  )s(k) 

A 

(3.4) 

eiD(k) 

''(k)  IL> 

'_1-t-(lA)^Td(k-1)P(k-1)0d(k-1) 

(3.5) 

P(k)  =  |p(k- 1 ) [-^7 ]  p(k‘  1  )ltd(k- 1  )ikTd(k- 1  )P(k- 1 ) 

A  A  e|Q(k) 

(3.6) 

The  identifier  is  chosen  to  represent  an  input-output  model  of  the  process  with  delay,  and 
(A.  1 1)  becomes 

td(k)=[^  bj(k-1-nd)q(,  +  nd)]u(k)+  [£]  a,(k-1-nd)q ']  yd(k)  (3.7) 

i=l  t=i 

where  td(k)  is  the  estimated  process  output  delayed  by  nd  sampling  instants. 

The  identification  error  becomes 

e|0(k)  =  yd(k)-td(k)=  (3.8) 

The  identifier  in  (3.7)  is  inverted  to  derive  the  control  law.  From  (3.7), 

td(k)  =  b)(k-1-nd)u(k-1-nd) 

n  a  n  A 

+  [£  bi(k-1nd)q  (l  +  nd)]u(k)+ a^k-l-n^q ']yd(k). 

i  =  2  is  1 

Solving  for  the  control  signal  u(k-nd)  that  would  have  been  required  to  force  the  one-step 
ahead  identifier  output  td(k+  1)  to  follow  the  reference  model  signal  leads  to 

1  n  A 

u(k  nd>  =  r:  :(xR(k)-[X]  bj(k-1-nd)q  (M  +  nd*3u(k)  (3.9) 

bl(k'nd)  i  =  2 

-[£  ai(k-1-nd)q'("1)]yd(k)} 

i=  1 

This  approach  specifies  what  the  control  signal  should  have  been  nd  sampling  instants  in  the 
past.  The  actual  control  signal  is  formulated  according  to  (3.9),  with  the  left-hand  side  of  the 
equation  replaced  by  u(k): 


u(k)  =  A— - -{xn(k)-LX;  b,(k  n(J)q  1  +  nu'J u(k)  [£  a^k-n^q ] yd(k)}  (3.10) 


bi(knd) 


3.3.2.  Control  Penalty 

The  control  penalty  is  reformulated  to  account  for  control  delays.  The  Z-transform  of  (3.10), 
with  the  control  penalty  included,  is 

£^{xB(k>.[£;  a(knd)z"’']yd(z')} 

u(z1)=  tVkrV  i=i 


[E  b,(k.na)2«''*V] 


and  the  controller  characteristic  equation  is 

F  (z')  =  u/(k)  [y]  b(kn  )z(M+nd)] 

b,<k  nd>  ' 


(3.11) 


The  Jury  conditions  are  applied  to  (3. 1 1)  for  the  case  of  n  =  2  and  nd  =  1.  Upon  applying  the 

transformation  z  =  yz\ 

.  .2  bJk-1) 

F  (z  )  =  z  +7r(k)-^ - £ 

d  b,(k-l) 

For  this  quadratic  equation,  the  Jury  conditions  lead  to 


l7r(k)b2(k)/b((k)l<y  . 


The  control  penalty  is  computed  as 


w(k)  = 


1  for  Ib^b^y2 


[bt(k)/b2(k)]y2  for  lb2/btl>y2 


(3.12) 


3.3.3.  Reference  Model 


The  reference  model  in  Figure  A-1  is  modified  in  Figure  3-3  to  include  nd  unit  delays  in  the 
output.  In  Figure  3-3,  wm0(ie|(z  ’)  is  specified  in  (A. 7),  and  the  root-locus  as  a  function  of  the 
gain  (for  nd  - 1)  is  shown  in  Figure  3-4.  There  are  two  dominant  right-half  z  plane  poles  and  a 
fast  pole  on  the  negative  real  axis.  The  gain  is  tuned  to  achieve  critical  damping  of  the  two 
dominant  poles.  The  critical  gain  is  found  as  follows.  The  characteristic  equation  is 


K  Per  1 1 


WM0„el(  2  ) 


/  Mode  r 


Figure  3-3:  Reference  Model  With  Delays 


Figu  re  3-4:  Reference  Model  Root-Locus  with  One  Delay 

b,z  +  b, 

1  +  KoT2 - *—  =  0  (3.13) 

Differentiating  (3.13)  with  respect  to  the  gain,  Kp  results  in  the  third  order  polynomial 
2b1z3  +  (3b2-a1b|)z2-2a)b2z-a2b2  =  0 

The  roots  are  found  with  the  Fortran  IMSL  [IMSL  82]  library  routine  ZPOLR,  and  the  root  lying 
between  1  and  -a2  corresponds  to  zb  .  The  critical  gain  is  determined  by  evaluating 
(3.13)  at  z  =  z.  • 


K  =  — fre 
Pcrit 


.  brenk'ai2bre 
blZt>reak  +  b2 


The  closed-loop  transfer  function  becomes 

.  b.z'+bz2 

M  .(z'1)  =  K  • - ; - - 2_; - ~ 

and  the  model  parameters  of  (A.  13)  are 


b,=b.KPch« 


b2  "  b2KPcrit 


b3  =  0 


a,*3, 


a2  =  a2blKPcrit 


a3  "  b2KPcrit 


m  =  3 


3.4.  Signal  Biasing  and  Gravity  Compensation 

To  apply  the  uncoupled  MRAC  controller  to  a  coupled  nonlinear  system,  two  modifications 
are  required: 

1 .  Biasing  of  the  reference  and  output  measurement  signals  so  that  the  operating 
point  is  numerically  zero,  and 

2.  Addition  of  gravity  compensation  if  the  plant  under  control  is  type  zero. 

3.4.1 .  Signal  Biasing 

To  assure  that  the  FP  steady-state  errors  go  to  zero,  the  control  operating  points  must  be 

referenced  to  zero.  To  demonstrate  this  requirement,  a  step  input,  r(k)  =  R,  is  applied  to  the 

controller  and  the  steady-state  FP  error  is  calculated.  In  steady-state,  it  is  assumed  that  the 

reference,  output,  and  control  signals  have  the  following  constant  values: 

XR(k)  =  R 

y<k>  =  Yss 
u(k)  =  14, 

In  addition  the  estimated  parameter  vector  is  assumed  to  be  in  steady-state: 

If  it  -  1,  the  controller  in  (A.  12)  or  (3.10)  reduces  to: 


Upon  solving  (3.14)  for  R.  the  steady-state  FP  error,  epp  S3  =  R-Y  ,  is 

«’s> 

i=  i 

A 

where  W(1),  the  estimated  DC  gain  of  the  equivalent  SISO  system,  is 


A  '  -  1 

W(1)  = - 


i=  1 


If  the  open-loop  plants  are  type  I  (ie;  there  is  an  open-loop  pole  at  z  =  1 ),  then  U  must  be  zero 

A  “ 

for  the  system  to  be  at  rest.  For  ess  =  0,  then  Y  /W(1)  must  be  zero.  Thus,  either  or 

A 

1/W(1)  must  be  zero.  Type  I  systems  have  infinite  open  loop  gains.  However,  simulations 
using  type  I  plants  with  nonlinear  gains  (Section  5.4)  produce  steady-state  equivalent  SISO 
model  estimates  which  can  be  type  0.  Type  0  processes  have  finite  open-loop  gains.  Thus, 
the  operating  point,  Y  ,  must  be  forced  to  be  zero.  To  accomplish  this,  the  reference  signal 
is  forced  to  zero,  and  the  output  signal  is  biased  by  the  reference: 

1.  XR(k)=0; 

2.  y(k)<— [y(k)-XR(k)j;  and 

3.  Initial  model  conditions:  x(0)«— [y(0)-x(0)]. 


3.4.2.  Gravity  Compensation 

Let  a  type  0  SISO  process  be  under  MRAC  control.  When  the  gravity  component  is  zero,  no 
actuating  force  is  required  and  =  0.  From  (3. 15),  the  measured  output  signal  must  then  be 
referenced  such  that  Y^  =  0  when  =  0.  In  contrast,  each  type  0  process  in  a  coupled 
system  (such  as  an  articulated  robot  arm)  has  an  equilibrium  point  which  is  configuration 
dependent.  The  zero  reference  point  cannot  be  assigned  in  advance.  To  overcome  this 
problem,  the  identifier  in  (3.7)  is  modified  to  include  the  bias  aQ  as  follows: 

n  a  n  A  A 

ld(k)  =  CX]  bi(k-1nd)q  (i  +  nd)]u(k)+  [£  a.(k- 1  -nd)q ']  y  (k)  +  a  (k-1-nd) 


(3.16) 
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The  information  and  parameter  vectors,  respectively,  are  redefined  to  be 

^(k-1)=[u(k-1 -nd)...u(k-n-nd)  1  yd(k-1)...yd(k-nd)]T  ([2n  +  1]  X  1)  inform,  vector, 
^T=(b1...bnaoa1...an)  ([2n  +  1)  X  1)  parameter  vector 

and  the  dimension  of  adjustable  gain  matrix  becomes  ([2n  +  1]  X  [2n  +  1]).  This  identification 
structure  has  been  used  successfully  by  Koivo  [Koivo  St  ],  for  optimal  least-squares 
estimation,  to  estimate  an  equivalent  SISO  model  for  a  robot  joint.  With  this  identifier,  the 
controller  in  (3.10)  becomes 

n  n 

U(k)  =  ^.  i-{x“(k)-[E  b(k-n  d)q(i  +  nci>]u(k)-[V  a  (k  n  )q  j]y  (k)}  (3.17) 

bi(knd)  i  =  2  i  =  1 

W-nJ 

For  control  of  type  0  processes,  a  steady-state  error  can  occur  when  a  control  penalty  and 
U  #0,  according  to  (A.  13).  Neuman  and  Morris  suggest  that  auxiliary  fixed  PSD  control  can 
be  used  to  drive  these  errors  to  zero.  Alternatively,  the  adaptive  bias  term  proposed  in 
(3.16)  can  also  drive  steady  state  errors  to  zero.  This  approach  requires  that  the  control 
penalty  not  be  applied  to  the  bias  term  in  (3.17).  Also,  when  ramp  inputs  are  applied  to  type  I 
processes  (e.g..  moving  targets),  steady-state  errors  could  also  be  reduced  to  zero  with  this 
approach. 

3.5.  Linear  Control  Interpretation 

When  a  SISO  linear  process,  under  adaptive  control,  remains  constant  and  the  adjustment 
mechanism  converges,  the  closed -loop  MR  AC  becomes  a  linear  time-invariant  feedback 
system.  This  is  called  linear  model  following  control  (LMFC)  [Landau  79].  When  the  actual 
plant  is  time-varying,  the  MRAC  attempts  to  perform  as  if  the  true  linear  parameters  were 
being  utilized,  since  the  nonlinear  and  time-varying  adjustment  mechanism  attempts  to  drive 
the  identification  error  to  zero.  The  characteristics  of  the  LMFC  approximate  those  of  the 
MRAC  even  when  the  nonlinear  and  time-varying  adjustment  mechanism  is  active.  Linear 
analysis  of  a  SISO  linear  autonomous  process  under  LMFC  control  (with  reference  model  and 
control  penalty  enhancements  included)  can  thus  provide  insight  into  MRAC  control. 
Neuman  and  Morris  have  performed  extensive  linear  analysis  to  systems  without 
measurement  delay.  This  section  extends  the  linear  root-locus  analysis  to  systems  with  delay. 


73 


This  analysis  shows  that  with  measurement  delays,  the  control  penalty  enhancement  is  a 
requirement  for  relative  stability,  and  the  stability  margin  of  the  SP  MRAC  system  exceeds  that 
of  the  FP  MRAC. 

In  the  following  analysis,  the  plant  under  control  is  assumed  to  be  linear  (described  by 

A 

(A. 8)),  and  the  adjustable  parameters  have  converged  to  their  true  values,  that  is  /£  =  /?. 

3.5.1.  FP  MRAC 

The  closed-loop  transfer  function  of  the  FP  MRAC  is  derived.  The  Z-transform  of  the 
controller  (3.10),  with  control  penalty  included,  is 


,  K  R  1  K  1 

U(z  )  =  — XR(z  ’)  + - f>-p — Y  ,(z  ) 

1-z  jl<  1  -z  dKu  d 


where  the  controller  gains  are 

K"(z ')±Ku=.^r  b.z(M) 

b  ^  1 

7i  (k)  '  1  =  2 

K= - 

bl  itt  1 


(3.18) 


) 

\  B,„(  z  ) 


z  '  A  „(  z  1 ) 


vR  /  *>  * 

X  (  z  ) 

~lio\  v' 


/v/  \y  / 


Y(  z  ) 


u(z-‘)  l  Yu(zM) 

\  -i  \ 


-  z  ”nu 


Figure  3-5:  FP  LMFC  Control 

From  the  block-diagram,  in  Figure  3  5,  the  closed-loop  transfer  function  is 
Y  (z1)  B  (z  ')  Kz’^Wtz'1) 


R(z  )  z  1 A  (z  1 )  (1-z  dKl,)-KyW(z  ')z 
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Upon  substituting  the  controller  gains  into  (3. 19),  the  closed-loop  transfer  function  becomes 

77  (k)  .  z  nd 

Y.(z’)  B  (z  ')  )— — 


Y  ,(z  ’)  B  (z  ') 
it  . ' _ or _ 

R(z  ')  A  (z'*) 


(3.20) 


n.  7r(k)  ,  z'nd 

[l-77(k)Z  (i]  +  —\N(z')—T 

b1  Z 


When  nd  =  0  and  7r(k)  =  1 ,  (3.20)  reduces  to 


Yd<zl>-  V1) 

R(z')  Am(z‘) 


M(z’) 


and  the  closed-loop  transfer  function  reduces  to  the  model  in  (A. 3).  If  nd*0  and  vr(k)  =  l, 
(3.20)  becomes 

1  .  z"\j 

Y  (z'1)  -W(z  ')— 

-^V  =  M(z-V  b.  2 

R(z')  - - - 

-n  1  ,  Z  d 

[1-z  d]  +  -W(z  ^ 
b,  z 

and  the  closed-loop  response  is  a  filtered  model  response.  Without  a  control  penalty,  this 
filter  can  be  unstable  as  demonstrated  by  the  following  root-locus  analysis. 


The  aiaracteristicf  equation  of  (3.20)  is 

n  77 (k)  ,  z'nd 

1  -  77  (k)z  d  + - W(z  ')-- T-  =  0 

b,  z’ 


(3.21) 


Let  the  process  under  control  be  described  by  (A. 6),  and  let  there  be  a  single  delay.  This 

structure  is  simulated,  in  Chapter  5,  to  verify  the  root-locus  analysis,  with  the  parameters 

"model  =  0  072  SeC 
Kmode,  =  36.58 
T  =  0.033  sec 

and  a  reference  model  with  two  closed-loop  poles  at  z  =  0.730,  and  one  at  z  = -0.015.  The 
roots  of  the  characteristic  equation,  in  (3.21),  as  a  function  of  the  control  penalty  77  (as  77 
varies  from  0  to  1)  are  shown  in  Figure  3-6.  When  77  =  0,  there  are  two  open  loop  poles  which 
originate  at  the  open-loop  process  poles 


zi  ~  e 


rm  =  .632  ;  and  z2  =  1 


and  a  pole  at  the  origin.  As  the  control  penalty  decreases  (and  77  approaches  1),  the  dominant 
poles  originating  from  the  process  poles  break  away  from  the  real  axis  (when  77^0.015), 
become  oscillatory  and  eventually  cross  the  unit  circle  and  become  unstable.  A  relatively 
large  control  penalty  is  thus  required  to  achieve  a  smooth  response,  at  the  expense  of  slower 
tracking  of  the  reference  model. 
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Figure  3-6:  FP  Root  Locus  vs.  ir 


3.5.2.  SP  MRAC 


The  Z-transform  of  the  SP  reference  model  is 
R/.-U  BmfcT)r 


Upon  substituting  (3-22)  into  (3.18),  the  Z-transform  of  the  SP  controller  is 

KBJz1)- 


U(z 


i)= _JL-.fr: 

’  1-z  dK'ilL 


!]R,zV[KU^]yd,z',}. 


From  the  block  diagram  in  Figure  3-7,  the  closed-loop  transfer  function  i: 


77 


1  7TZ  d  W(z') 

,  b,  1-17Z  d  z  1 

Yri(z')  ,  1 

-d  -  =  B  (z ')• - 

i  tn'  ' 


R(z  ) 


(3.23) 


,  1  wz  d  Wfz1) 
1  +  AJZ  )— T - T r- 


b,  1-7TZ 


When  nd  =  0  and  7r(k)  =  1 ,  (3.23)  reduces  to 

Y.(z’)  B  (z1)  . 

=  ~  M(z  ') 

R(z')  Am(z') 

and  the  closed-loop  model  reduces  to  the  model,  as  predicted.  If  nd*0  and  7r(k)  =  1, 
(3.23)  reduces  to 


1  z  d  W(z') 


^  =  B  (z  V 

R(z  )  m 


b,  1-z  ° 

•n 

1  Z  ' 

1  +  A  — - 1 

mb,  1-z 


(3.24) 


W(z'1) 


which  does  not  reduce  to  the  model.  To  understand  the  characteristics  of  (3.24),  the  root- 
locus  is  exanrned.  The  characteristic  equation  of  (3.24)  is 


.  1  '<rz  d  W(z  ’)  ^ 

1  +  A - *t~: - r—  =  0. 

1  7TZ  ,j  z 


(3.25) 


For  the  process  pararnaters  in  Section  3.5.1,  the  roots  of  the  characteristic  equation,  in  (3.25), 
as  a  function  of  the  control  penalty  (ir  varies  from  0  to  1)  are  shown  in  Figure  3  8.  When  w  =  0, 
two  open-loop  poles  originate  at  the  process  poles,  and  there  are  three  poles  at  the  origin.  As 
the  control  penalty  decreases,  the  process  poles  migrate  toward  the  two  dominant  critically 
damped  model  poles  and  eventually  break  away  with  a  relatively  small  overshoot.  One  of  the 
poles  at  the  origin  migrates  toward  the  fast  model  pole  on  the  negative  teal  axis.  Even  though 
the  system  characteristics  tend  toward  the  model,  two  additional  poles  at  the  origin  break 
away  from  the  real  axis  causing  significant  oscillation  as  n  decreases. 

Comparison  of  the  SP  and  FP  root-loci,  in  Figures  3-8,  and  3-6  shows  that  the  SP  MRAC 
system  has  a  significantly  larger  margin  of  stability  when  operating  with  the  same  control 
penalty.  Since  it  is  desirable  to  minimize  the  control  penalty  (to  keep  the  closed-loop 
bandwidth  large),  the  SP  MRAC  is  recommended  for  control  of  syst;ms  with  delay. 


ROOT-LOCUS 

Figure  3-8:  SP  Root  Locus  vs.  it 

3.6.  Parameter  Specification 

Selection  of  MRAC  controller  parameters  involves  compromises  among  several  factors.  In' 
this  section,  guidelines  and  tools  are  presented  to  aid  the  control  engineer  in  making  suitable 
choices.  Optimal  selections  result  from  the  tuning  of  the  parameters  over  sped'  c  regimes  of 
system  operation.  Parameters  are  selected  in  Chapters  5  through  7  to  achieve  acceptable 
operation  over  a  wide  range  of  control  tasks,  as  opposed  to  optimizations  over  local  regions. 

3.6.1 .  Model  Time  Constant  Specification 


The  reference  model  specifies  the  desired  closed-loop  system  performance.  Control 
performance  can  be  defined  by  time  or  frequency  domain  specifications,  performance  index 
optimization,  or  pole-zero  location.  For  step-inputs,  a  convenient  performance  measure  is 
rise-time,  tf,  defined  as  the  time  it  takes  the  output  to  go  from  0  to  99%  of  it’s  final  value.  The 
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rise-time  criterion  is  specified  by  s-plane  pole  requirements  which  are  then  mapped  into  the 
z-plane  according  to  z  =  est.  The  model  time  constant.  rmode|  in  (A. 6),  is  then  selected  to  place 
zbreak  at  tlie  desired  focation  according  to  the  methods  outlined  in  Appendix  A,  for  nd  =  0,  or  in 
Section  (3.3.3),  for  nd  =  1 . 


An  s-plane  model  of  a  system,  with  a  pair  of  critically  damped  poles  at  un,  is 
X(s)  =  <on2 
Ref(s)  (s  +  wn)2 


where  wn  specifies  the  system  bandwidth.  The  unit  step  response  is 
x(t)  =  1-e“n'(1-wnt) 

The  bandwidth  vs.  rise-time  is  found  by  solving 

0.01  =ewn,r(1-wntr)  (3.26) 

Once  the  desired  rise-time  is  selected,  (3.26)  is  solved  numerically  for  «n.  This  bandwidth 
then  specifies  the  closed-loop  reference  model  pole  at 


Many  constraints  must  be  considered  in  specifying  tf.  Selecting  too  large  or  too  small  a 
rise-time  can  result  in  the  following  problems: 

1.  The  sampling  period  (in  Section  3.6.2)  places  constraints  on  u>n  and  thus  tf.  A 
lower  bound  on  tf  results  from  lower  bound  constraints  on  T. 

2.  As  tr  decreases,  <an  increases  and  zbreak  decreases.  As  zbreak  becomes  smaller, 
and  the  system  speeds-up,  the  the  controller  gain  increases,  thus  amplifying 
noise  and  degrading  system  performance.  As  the  controller  gain  increases,  the 
system  tends  to  saturate  more  which  in  turn  reduces  relative  stability.  As  trjse 
increases  and  zbrenk  approaches  the  unit  circle,  the  full-parallel  error  eFp 
(A. 23)  increases,  relative  to  any  identification  error  since  AJz  ')  increases. 


3.6.2.  Sampling  Period  Selection 

While  there  are  guidelines  for  the  selection  of  the  sampling  period  T,  the  choice  can  be 
forced  upon  the  designer  by  hardware,  software,  or  cost  constraints.  When  this  occurs, 
control  performance  criteria  must  be  evaluated  in  light  of  these  constraints. 


The  following  guidelines  are  used  in  to  specify  the  sampling  period  and  control  performance 
specifications. 
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3.6.2. 1 .  Fundamental  Upper  Bound 


When  the  closed-loop  system  bandwidth  is  specified  to  be  con.  the  Nyquist  sampling  theorem 
sets  the  upper  bound  on  the  sampling  period  according  to 

IT 


'UB 


(3.27) 


If  hardware,  software,  or  cost  constraints  limit  the  sample  period  to  Tconstfajnt,  where 

^"constraint 

the  closed  loop  system  bandwidth  must  be  constrained  by 
w 

u  < - . 

n  r 

constraint 


It  is  important  to  note  that  there  is  a  distinction  between  the  closed-loop  bandwidth  and  the 
highest  frequency  components  of  the  open-loop  process  dynamics,  since  these  two 
frequencies  can  be  quite  different.  Thus,  potentially  slow  visual  sampling  rates  can  be  used  to 
control  fast  open-loop  processes,  if  the  closed-loop  bandwidth  is  reduced  proportionally. 


3. 6. 2. 2.  Performance  Margin 

For  reasons  outlined  below,  TUfcJ  is  judged  to  be  insufficient.  A  performance  margin  (PM) 
can  be  specified  to  decrease  the  sample  period  according  to 

T__UB  where  PM>1 
PM 

and  therefore, 

IT 

T  < - . 

PM*  u 

n 

For  control  of  constant  systems,  typical  values  for  PM  are  2<PM<10,  depending  upon  the 
particular  application.  Adaptive  control  of  nonlinear  systems  require  larger  margins  (Section 
3.6.2.3).  In  terms  of  a  sampling  rate  to  closed-loop  bandwidth  ratio,  this  requirement  is 
f  1/T 

_a_  = - =  2*  PM.  (3.28) 

fBW  <V2w 

The  performance  margin  is  specified  to: 

1.  Reduce  any  delay  between  a  reference  signal  command  change  and  system 
response  to  that  change; 

2.  Smooth  the  system  response  to  the  control  inputs  which  are  applied  to  the 
process  through  a  DAC; 


3.  Minimize  aliasing  effects,  since  the  actual  system  may  have  attenuated  signal 
components  above  wn;  and 


4.  Increase  relative  stability  for  MRAC  with  discrete  delays  by  minimizing  the 
difference  between  the  estimated  control  signal  and  the  required  control  signal. 


From  (3.28),  the  constraint  on  the  closed-loop  system  bandwidth  becomes 

v 

u>< - 

n  PM*T  ,  .  , 
constraint 

3. 6. 2. 3.  MRAC  Identifier  Requirements 


(3.29) 


The  discrete  time-identifier  used  by  MRAC  places  additional  requirements  on  the  sampling 
period. 

1.  If  the  open-loop  plant  to  be  identified  is  constant  and  linear  with  no  complex 
poles,  then  no  additional  requirement  is  placed  on  T  (assuming  the  computer  has 
sufficient  precision  to  measure  smaller  input-output  signal  changes,  as  T 
increases  and  t  decreases).  The  plant  dynamics  can  be  identified  independently 
of  T.  If  the  plant  has  complex  poles  at  <i±ju)  ,  then  the  sampling  theorem 
requires  that 


If  sampling  period  constraints  cannot  satisfy  this  requirement,  then  an 
independent  velocity  feedback  (which  samples  at  a  higher  rate  than  the  main 
control  loop)  can  be  incorporated. 

2.  If  the  plant  is  time- varying  or  nonlinear,  then  the  sampling  period  must  be  chosen 
to  be  short  enough  to  track  parameter  variations.  In  evaluations  in  Chapters  5,  6, 
and  7,  performance  margins  of  10<PM<25  are  demonstrated  to  be  suitable  for 
tracking  the  nonlinear  plant  gains  for  IBVS  tasks  which  require  small  corrective 
motions.  Motions  over  large  distances,  and  thus  larger  gain  changes,  require 
even  higher  margins.  These  requirements  are  discussed  in  Chapters  5  and  6. 


3.6.3.  Adaptive  Gain  Matrix  Initialization  and  Limiting 

The  adaptive  gain  matrix  P(k)  in  (3.6)  is  initialized  to  a  non-zero  value  prior  to  the  start  of 
control.  Subsequent  control  tasks  can  then  use  previously  learned  values.  To  the  best  of  the 
author’s  knowledge,  guidelines  for  the  initialization  are  not  presented  in  literature  and  the 
approach  seems  to  be  empirical.  This  topic  is  an  area  requiring  future  research  since 
initialization  of  P(k)  will  affect  the  identifier's  rate  of  convergence  and  thus  the  controller’s 
relative  stability.  The  following  tools  and  guidelines  have  been  developed  to  facilitate  the 
experimentation. 

A 

I.When  the  estimated  process  gain  b,  is  small  it  becomes  difficult  to  find  an 
initialization  which  is  not  too  sensitive.  Thus,  the  reference  and  output  signals  are 
multiplied  by  a  §cale  factor  (SF)  to  increase  the  apparent  process  gain.  The  scale 
factor  is  increased  until  b,  execceds  0.001  . 
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2.  Since  the  operating  points  vary  widely  from  one  control  task  to  the  next,  it 
becomes  difficult  to  find  an  initialization  that  is  suitable  over  a  wide  range  of 
control  tasks.  Signal  biasing  (Section  3.4.1)  makes  it  much  easier  to  find  values 
which  can  be  used  over  a  wide  range  of  tasks. 

3.  In  accordance  with  Neuman  and  Morris'  work  [Neuman  79b],  P(0)  is  initialized  to 
be  a  diagonal  matrix. 

4.  When  the  sampling  interval  is  decreased,  P(0)  is  also  decreased. 

5.  Values  of  P(0)  found  to  be  suitable  for  control  of  uncoupled  SISO  systems 
appeared  to  be  suitable  for  control  of  the  coupled  systems. 

6.  When  P(0)  is  initialized  with  unsuitably  small  values,  the  controller  would  exhibit 
poor  stability.  In  these  cas^s,  if  the  tf^e  signs  of  the  steady-state  estimated  gain 
terms  were  opposite  (e.g.,  /?(>0.  and  (i2< 0),  then  larger  gain  matrix  initializations, 
resulted  in  improved  stabilty  and  consistent  signs  (Appendix  D). 


Equally  important  is  gain  matrix  limiting.  If  the  system  reaches  steady-state  (<t>  =  0),  the  gain 

matrix  adjustment  mechanism  reduces  to 
P(k)  =  (1/\)P(k  1), 

and  the  gain  matrix  thus  becomes  unbounded  when  \<1.  This  presented  implemention 
problems  for  Neuman  and  Morris  due  to  arithmetic  overflow  in  the  .u-processor.  To  overcome 
this,  Neuman  and  Morris  simply  reset  P(k)  =  P(0)  when  the  diagonal  element  Pnn(k)  exceeded 
a  thieshold  P.  .  .  In  this  dissertation,  simulations  are  implemented  on  a  large  mainframe 
where  overflow  is  not  an  immediate  problem.  However,  as  P(k)  becomes  large,  the  adjustment 
mechanism  becomes  too  sensitive  to  adaptation  to  changes  in  the  plant  parameters,  and 
decreases  system  relative  stability.  To  overcome  this  problem,  P(k)  is  reset  to  P(0)  whenever 

Pl.lW  0rP2n,2n<k)>P.hresh' 

where  the  threshold  is  selected  experimentally.  Similar  to  the  initialization  of  P(k),  the 
threshold  should  decrease  as  the  the  sampling  period  decreases. 


3.6.4.  Parameter  Vector  Initialization 

A 

While  hyperstability  does  not  put  any  constraints  on  the  initialization  of  £(0),  better 
initialization  leads  to  faster  convergence  of  the  identifier.  If  the  initialization  of  b1  is  too  large, 
the  rate  of  convergence  and  system  response  are  unacceptably  slow.  This  is  observed  to  be 
more  pronounced  as  the  reference  model  bandwidth  decreases.  If  b t  (0)  is  too  small,  then  the 
system  accelerates  rapidly,  decreasing  relative  stability. 
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Any  a’priori  knowledge  of  the  system  should  be  used  to  guide  the  initialization.  For  instance, 
as  the  sampling  period  becomes  smaller,  the  gains  {b.(}  become  smaller,  and  the  poles 
approach  unity.  Initializations  which  are  satisfactory  for  control  of  SISO  uncoupled  plants 
appear  to  be  acceptable  for  the  coupled  plant  initializations. 

3.6.5.  Weighting  Factor 

The  least-squares  weighting  factor  \  is  chosen  as  a  compromise  between  sensitivity  to 
adaptation  to  process  changes  and  over  reaction  to  noise  measurement.  For  the 
experimentation  presented  in  this  dissertation,  and  for  Neuman  and  Morris'  research,  \  =  0.85 
provided  satisfactory  response. 

3.6.6.  Control  Penalty 

The  controller  pole  bound,  y,  which  indirectly  sets  the  control  penalty,  is  tuned  to  minimize 
ringing  in  the  output.  Faster  model  responses  cause  increased  saturation  tending  to  ring  the 
controller  more.  When  this  occurs,  the  control  penalty  is  increased  by  reducing  y. 

Physical  implementation,  by  Neuman  and  Morris  [Morris  81],  of  MRAC  position  control  of  a 
DC  motor  required  smaller  values  of  y  than  for  simulation  trials.  This  resulted  from  the 
computational  delay  required  to  actually  compute  u(k).  The  LMFC  analysis,  in  Section  3.5, 
explains  why  systems  with  delay  require  increased  control  penalties.  Since  the  computational 
delay  is  only  a  fraction  of  the  sampling  interval,  Neuman  and  Morris  set  y  in  the  vicinity  of  0.9  . 
When  unit  delays  are  present,  as  with  IBVS,  larger  control  penalties  are  needed. 

3.7.  Summary 

In  this  chapter,  the  independent  MDAC  controller  is  extended  to  include  measurement 
delays.  Analysis  of  this  controller,  for  control  of  a  constant  linear  plant,  predicts  the 
requirement  for  a  control  penalty  to  achieve  relative  stability.  Chapter  5  includes  simulation 
examples  which  verify  the  algorithm  for  control  of  SISO  plants.  Analysis  of  independent 
control  of  a  coupled  nonlinear  plant  shows  an  additional  requirement  for  signal  biasing  and 
gravity  compensation  to  assure  zero  steady-state  errors.  Guidelines  for  the  practical  selection 
of  controller  parameters  are  emphasized  and  will  be  used  in  the  following  chapters  where 
actual  values  are  selected. 
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Chapter  4 

Task  Modeling  and  Simulation 

4.1 .  Overview 

The  purpose  of  this  chapter  is  to  specify  the  visual  servoing  task  configurations  which  are 
evaluated  under  fixed  and  MRAC  control  in  this  dissertation.  Mathematical  models  of  robot 
kinematics  and  dynamics,  camera  picture  taking,  and  feature  transformations  (which  are 
required  for  implementation  of  the  task  simulations)3  are  developed.  The  task  configurations 
are  selected  so  that  the  delay,  nonlinear,  and  coupling  aspects  of  IBVS  can  be  studied  and 
evaluated  independently  as  applied  to  control  progressively  complex  dynamic  and  kinematic 
systems  consisting  of: 

■*  1  OOF  linear  dynamic  and  kinematic  robot,  with  either  constant  or  nonlinear 
feature  sensitivity; 

•  2  DOF  linear,  uncoupled  dynamic  and  kinematic  robot,  with  nonlinear  and 
coupled  feature  sensitivity; 

•  2  DOF  nonlinear,  coupled  dynamic  and  kinematic  robot,  with  nonlinear  and 
coupled  feature  sensitivity;  and 

•  3  DOF  linear,  uncoupled  system  dynamics,  with  a  combination  of  linear  and 
nonlinear  kinematics,  and  nonlinear  and  coupled  feature  sensitivity. 

The  1  DOF  configuration  simulations  are  summarized  in  Chapter  5  to  verify  the  basic  MRAC 
algorithm  (with  and  without  delay),  and  provides  an  initial  evaluation  cf  a  system  for  which  the 
control  variable  is  nonlinearly  related  to  the  physical  plant  variable.  The  2  DOF  systems,  which 
include  coupled  feature  transformations,  are  evaluated  in  Chapter  6.  For  these 
configurations,  the  basic  MRAC  algorithms  will  be  augmented  by  a  hierarchical  control 
strategy,  and  feature/joint  assignment  is  based  on  the  c  incept  of  diagonal  dominance.  One 
of  the  2  DOF  configurations  represents  an  ideal  "look  and  move"  structure  since  the  robot  is 

3 

All  simulations  are  implemented  on  a  DEC  20  processor. 
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dynamically  and  kinematically  linear  and  uncoupled.  The  other  configuration  specifies  an 
articulated  robot  arm  in  order  to  investigate  the  relative  contributions  of  nonlinear  and 
coupled  robot  dynamics  and  kinematics.  The  3  DOF  simulation  represents  a  visual  servoing 
configuration,  with  planar  Cartesian  and  rotational  degreesof-freedom,  presently 
implemented  in  the  CMU  Flexible  Assembly  Laboratory  [Sanderson  83b]  using  a  static 
position-based  "look  and  move"  control  structure.  Simulations  of  this  configuration,  with 
linear  and  uncoupled  dynamics,  emphasizes  the  feature  selection  process.  It  is  natural  to 
implement  IBVS  for  this  system  in  future  research. 

This  chapter  is  organized  as  follows.  In  Section  4.2,  the  task  configurations,  including 
transducer  mounting  (the  transducer  is  assumed  to  be  a  camera),  robot  type,  and  object  in 
the  scene  are  described.  In  Section  4.3,  the  dynamic  modeling  and  simulation  of  the  linear 
and  uncoupled  robots  is  presented.  The  dynamic  and  kinematic  modeling  and  simulation  of  a 
2  DOF  robot  arm  is  described  in  Section  4.4.  Modeling  of  picture  taking  and  feature 
measurement  is  described  in  Section  4.5. 

4.2.  Task  Configurations 

This  section  describes  the  visual  servoing  task  configurations  (i.e.,  transducer  mounting, 
robot  type,  and  object  in  scene).  The  first  configuration  describes  a  2  DOF  system.  This 
system  is  then  used  to  simulate  1  DOF  systems  by  restricting  relative  motion  to  one  of  the 
degrees-of  freedom. 

4.2.1 .  Configuration  1 :  2  DOF  with  linear  kinematics  and  dynamics 

The  camera  is  mounted  on  a  2  DOF  linear  kinematic  system  such  as  a  Cartesian  robot,  a  set 
of  translational  stages,  or  possibly  a  mobile  robot.  The  camera  is  servoed  relative  to  a  fixed 
line  in  space  to  control  the  relative  position  between  the  robot  and  a  line  (Figure  4- 1 )  .  A  line  in 
space  problem  is  selected  because  the  corresponding  sensitivity  functions  can  be  easily 
analyzed.  It  is  complex  enough  to  be  both  interesting  and  useful.  The  line  is  assumed  to  be 
fixed  in  spaced  so  that 

1.  The  line  lies  entirely  in  the  plane  defined  by  [ XQ ,  Yo];  and 

2‘  Zrel  =  0 

The  camera  is  mounted  to  the  robot  so  that: 

1  [Ycam]  is  parallel  to  [YJ;  and 


(  10  --  '  <nch  ) 


Zo 

[X  ,Y  ,Z  ]  World  Coordinate  Frame 

1  o’  o  cr 

[X„  „,.Y  „  ,Z _ ]  Camera  Coordinate  Frame 

1  cam  cam  camJ 

I  3  dimensional  length  of  Jme 

0  . .  Angle  of  line  relative  to  X 
obi  o 

X(t),Y(t)  Position  of  line  relative  to  world 

X  ,Y  K  Position  of  center  of  line  relative  to  world 
obi  obi 

X  ,(t),Y  ,(t  Position  of  line  relative  to  camera 

rel'  '  rel 

Figu  re  4- 1 :  Configuration  1 

2  f^caml is  Paral,el  t0  [Z0] 

The  robot  is  modeled  as  X-Y  translational  stages  with  the  Y  stage  mounted  on  top  of  and 
orthogonal  to  the  X-stage.  The  camera  is  fixed  to  the  Y  stage.  The  X-stage  controls  only  x(t) 
and  the  Y  stage  controls  only  y(t).  The  actuators  are  armature  controlled  DC  motors  with 
negligible  armature  inductance.  When  the  translational  stages  are  orthogonal,  the  axes  are 
decoupled.  In  addition,  it  is  assumed  that  gravity  acts  in  the  Z-axis  direction  so  that 
gravitational  effects  are  abscent.  Thus,  the  robot  is  modeled  an  uncoupled  linear  system. 


It  is  shown,  in  Chapter  5,  that  when  motion  is  constrained  to  the  X-axis,  and  the  center-of- 
gravity  of  the  line  image  is  chosen  as  the  control  feciture,  this  configuration  becomes  a  1  DOF 


autonomous  linear  system.  When  motion  is  constrained  to  the  Y-axis,  and  the  line  image 
length  is  the  control  feature,  then  the  configuration  becomes  a  nonlinear  system.  When  both 
degrees-of- freedom  are  free,  the  robot  represents  an  idealized  incremental  robot  system 
(Section  2.3.2. 1),  since  the  dynamics  and  kinematics  are  uncoupled  and  linear.  Thus,  the 
resulting  control  structure  represents  a  dynamic  "look  and  move"  structure. 

4.2.2.  Configuration  2:  2  DOF  with  nonlinear  kinematics  &  dynamics 

In  the  second  configuration,  the  camera  is  mounted  to  a  2  DOF  nonlinear  kinematic 
articulated  robot  arm  (Figure  4-2). 


do  =  1  'nch ) 


r1  ,r2  Length  of  robot  links 
<?1(t),02(t)  Link  angles 

/9„(t)  Angle  of  X  relative  to  the  second  link 
3'  '  3  cam 

Figure  4-2:  Configuration  2. 

The  line  is  fixed  in  space  so  that: 

1.  The  line  lies  entirely  in  the  plane  defined  by  [Xo,YJ;  and 


The  camera  is  mounted  so  that 


AD-A150  057 


UNCLASSIFIED 


DYNAMIC  VISUAL  SERVO  CONTROL  OF  ROBOTS:  AN  ADAPTIVE 
IMAGE-BASED  APPROACH! U)  CARNEG I  E-MELLON  UNIV  PITTSBURGH 
PA  ROBOTICS  INST  L  E  NEISS  APR  84  CMU-RI-TR-84-16 


1.  #3(t)  is  automatically  rotated  to  \0  ((t)  +  02(t)]  so  that  [Y  a  ]  is  always  parallel  with 

rv 

2.  t^caml iS  Paral,el  to  tz0l;  and 

-A  -u 

3.  The  end  of  link  2  is  coincident  with  the  origin  of  [X  ,Y  tZ _ 1 

cam  cam  cam 

The  robot  is  modeled  as  a  two  linkage  arm,  with  nonlinear  and  coupled  dynamics.  The 
actuators  are  armature  controlled  DC  motors  with  negligible  aramature  inductance.  It  is 
assumed  that  gravity  acts  in  the  Z  direction  so  that  gravitational  effects  are  abscent. 

4.2.3.  Configuration  3:  3  DOF  with  nonlinear  kinematics,  and  linear  and  uncoupled 
dynamics 

This  visual  servoing  task  consists  of  servoing  a  three-dimensional  object  (described  in 
Section  4.5. 2.2)  relative  to  a  stationary  camera.  The  object  is  placed  on  a  3  DOF  linear 
kinematic  manipulator  consisting  of  the  set  of  X-Y  translational  stages,  described  in 
Configuration  1 ,  and  an  additional  rotational  stage  mounted  on  top  (Figure  4-3). 

The  camera  is  mounted  relative  to  the  X-Y-0  stages  so  that: 

1  t*caJ is  parallel  to  [XJ; 

2.  The  plane  defined  by  P^^.Y  ]  is  coplanar  with  the  plane  defined  by  [ZQ, Yq]; 

and 

3.  The  plane  defined  by  [Xturn,Yturn]  is  coplanar  with  the  plane  defined  by  [X0,Yo]. 

The  dynamics  of  this  system  are  simplified  by  assuming  that  the  inertial  coupling  and 
centrifugal  forces  caused  by  the  load  are  negligible.  This  assumption  is  justified  when  the 
load  does  not  shift  the  center-of-mass  away  from  the  center  of  the  turntable.  The  center-of- 
mass  will  remain  in  the  vicinity  of  the  turntable  center  when  the  load  is  centered  on  the 
turntable  or  has  a  weight  much  smaller  than  that  of  the  turntable  platter.  The  manipulator  is 
then  modeled  as  a  dynamically  linear  and  uncoupled  system. 

The  evaluation  of  this  configuration,  in  Chapter  7,  focuses  on  the  feature  selection  process, 
by  evaluating  system  performance  as  a  function  of  the  coupling  index  D(JWp)  for  various 
candidate  feature  subsets.  Since  D(JWp)  is  the  measure  of  overall  system  coupling,  system 
performance  is  evaluated  at  the  joint  level  (i.e.,  performance  of  the  X-Y  6  axes).  The  3  DOF 


configuration  can  be  viewed  as  a  Cartesian  robot  with  the  turntable,  or  0-stage,  acting  as  the 
final  link  (or  gripper).  The  object,  which  sits  on  the  turntable,  is  viewed  as  a  tool  held  by  the 
gripper.  The  camera  is  stationary  relative  to  a  world  reference  frame.  The  motion  of  the 
object  is  not  uncoupled  relative  to  the  world,  because  radial  points  on  the  0-stage,  upon 
which  the  objects  rests,  are  coupled  to  both  X  and  Y  world  coordinates.  Therefore,  J*Jfeat. 
and  this  is  not  a  "look  and  move”  system  with  respect  to  the  object’s  frame.4  While  the  final 
position  and  orientation  of  the  object  relative  to  the  world  can  be  controlled,  the  object  path 
may  not  correspond  to  the  X-Y  table  trajectory,  unless  the  object  is  centered  on  the  0-stage.  It 
is  of  practical  importance  to  be  able  to  control  the  X-Y-0  stage  trajectories  since  their  motion 
must  ultimately  be  constrained  within  the  boundaries  of  a  physical  workcell  environment. 

4.3.  Plant  Modeling  for  Translational  and  Rotational  Stages 

The  X-Y-0  stage  dynamics  are  modeled  in  this  section.  By  neglecting  dynamic  interactions, 
each  stage  can  be  modeled  independently.  Newton’s  Second  Law  leads  directly  to  the 
dynamic  equations  of  motion. 

4.3.1 .  Rotational  Stage 

The  rotational  stage  (in  Configuration  3)  consists  of  a  turntable  platter  driven  through  a 
gear-box  by  a  DC  motor  with  negligible  aramture  inductance.  An  equivalent  diagram  is  shown 
in  Figure  4-4. 

The  following  assumptions  are  made  to  develop  the  mathematical  model: 

1.  Ideal  gearing  (i.e.,  negligible  backlash,  friction,  and  inertia); 

2.  Negligible  turntable  bearing  friction; 

3.  No  motor  voltage  off-sets;  and 

4.  Amplifier  (amp)  bandwidth  (BW)  is  large  in  comparison  with  the  motor/turntable 
mecahanical  BW;  thus,  the  amplifier  dynamics  can  be  neglected  in  the  modeling. 

The  transfer  function  Gm(s),  from  the  applied  voltage  efl(t)  to  the  motor  shaft  displacement 
0m(t)  is  [Kuo  82] 

4This  configuration  would  represent  a  "look  and  move"  system  if  the  task  were  configured  such  that  the  camera 
was  mounted  and  centered  on  the  0-stage,  while  observing  a  stationary  object. 
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Figu  re  4-4:  Rotational  Stage  Model 
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where  the  equivalent  motor  gain  constant  K  is 
K 

K= - 1 - 

m  R_a_  +  K.K. 


am  t  b 


and  the  equivalent  motor  time  constant  r  is 

f  .  R*W. 
m  R_a_  +  K.K. 


am  lb 


and  the  equivalent  inertia,  I  ,  referenced  to  the  motor  shaft  is 
leqa|m  +  ,l/ll2 


applied  control  signal  voltage 

(volts) 

unity  gain,  high  B.W.,  high  impedance  amplifier 

(volts/volt) 

armature-winding  resistance 

(ohms) 

armature  current 

(amps) 

back  E.M.F. 

(volts) 

angular  displacement  of  motor  shaft 

(radians) 

motor  torque 

(oz-in) 

armature  moment  of  inertia 

(oz-in-sec2) 

motor  viscous- friction  coefficient 

(oz-in/rad/sec) 

gear  ratio  N2/N? 

(motor/turntable) 

angular  displacement  of  turntable 

(radians) 

turntable  torque 

(oz-in) 

moment  of  inertia  of  turntable  +  load 

(oz-in  sec2) 

hack  emf  constant 

(volts/rad /sec) 

motor  torque  constant 

(oz-in/amp) 

(4.1) 


(4.2) 


(4.3) 


4.3.2.  X-Y  Stages 


A  translational  stage  (in  Configurations  t  and  3)  translates  the  load  through  a  lead-screw 
mechanism.  An  equivalent  diagram  is  shown  in  Figure  4-5. 
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Figure  4-5:  Translational  Stage  Model 
The  following  additional  assumptions  are  made  to  develop  the  model: 

1.  The  lead-screw  mechanism  has  zero  backlash  and  negligible  friction;  and 


2.  Stage  bearing  friction  is  negligible. 


The  inertial  equivalent,  lw,  of  the  load  referenced  to  the  motor  is  [Electro-Craft  80] 

W(lbs)  2 

'w* — § —  x  -0010417  (oz-in-secz) 

When  the  equivalent  inertia,  l»  I  + 1, «,  +  L.  is  substituted  into  (4.3),  the  transfer  function 

6q  m  lo  w 

model  structures  for  the  translational  stages  are  equivalent  to  the  rotational  stage  model.  The 
transfer  functions  G_  (s)  and  G„  „(s),  from  the  applied  aramature  voltages  ejt)  to  the  X  and 

iii-x  m*y  a 

Y  axes  motor  shaft  displacements,  respectively,  are 


m-y 


s<V+,) 


4.3.3.  X-Y-Theta  Stage  Parameters 

The  X-Y -6  stage  model  parameter  values  are  based  on  commercially  available  products 
such  as  the  AEROTECH  stages  used  in  the  Robotics  Institute  Flexible  Assembly 
Lab  [Sanderson  83b].  The  system  includes  a  model  316  rotational  (Table  4-3),  and  two  model 
416  translational  stages  (Table  4-2).  Each  stage  is  driven  by  a  motor  which  is  equivalent  to  an 
ELECTRO-CRAFT  E-508  (62  oz  in)  DC  motor  (Table  4-1). 

I  =0.0015  oz-in-sec2 

m 

a  =  0.00095  oz-in/rad/sec 
m 

Ra  =  3.36fl 

Kb  =  0.037  volts/rad/sec 
K,  =  5.20  oz-in/amp 
Volts(max)  =  40 

Table  4- 1 :  Electro-Craft  E-508  DC  MOTOR 
Net-weight  =  50  lbs 
Platter  weight  =  10  lbs 
Lead-screw  type  *  Steel  Ball-screw 
Lead-screw  pitch  =  7  threads/in 

Lead-screw  dimension  =  16  x  .55  in 
Table  4-2:  Aerotech  416  Translational  Stage 

Net- weight  =  29  lbs 
Platter  weight » 10  lbs 
platter  diameter  =  1 0  in 
gearratio  (ij)  =  108:1 

Table  4-3:  Aerotech  316  Rotational  Stage 

It  remains  to  calculate  the  equivalent  inertial  values  for  each  stage.  It  is  assumed  that  the 
load  inertia  equals  the  turntable  platter's  inertia.  This  assumption  is  not  critical  because  of 
the  linear  and  uncoupled  assumption  and  the  large  gear  ratio  (tj  =  108). 


The  equivalent  inertia  for  the  0-stage  is 


The  platter  is  modeled  as  a  thin  disc  so  that 
Wflbsjr^in) 


turntable-platter  ~ 


(oz-in-sec  ) 


10x(5)z  2 

- - =  5.208  (oz-in-sec2) 

48 

Since  l|oajj  -  turntable-platter' 

I  g  =  0.0015  +  (1  /108)2(2x5.208)  =  0.0024  oz-in-sec2  (4.6] 

The  equivalent  inertia  for  the  Y-stage  is  calculated  according  to 

leq-Y  =  lm  +  lLS+,W 

The  lead-screw  inertia  is 

lLS  =  D4(in)xlength(in)x(0.001 1667) 

=  (.55)x16x(0.001 1667)  =  0.0017  oz-in-sec2 

For  a  10  lb.  load,  the  net  load  on  the  V-stage  is  49  pounds  [i.e.,  0stage(29)  +  Y-platter(lO)  + 

load(IO)  ].  Thus, 

1 

lw  =  49x^x0.001 041 7 

=  0.001  oz-in-sec2 
Therefore, 

I  v  =  0.001 5  +  0.001 7  >  0.001  =  0.0042  oz-in  sec2  (4.7) 

The  net  load  on  the  X-stage  is  99  pounds  [i.e.,  0-stage(29)  +  Y-stage(50)  +  X-platter(lO)  + 

load(lO)].  Thus,  the  equivalent  inertia  of  the  X-stage  is, 

1 

L,  =  99x^x0.001 041 7 


=  0.0021  oz-in  sec2 


Hence, 


I  v  =  0.0015  +  0.0017  +  0.0021  =0.0053  oz-in-sec2  (4 

cQ'A 

Upon  substituting  (4.6),  (4.7),  and  (4.8)  into  (4.3),  the  equivalent  model  parameters  become 


T  a  =  0.041  sec 

m*0 

r  =  0.072  sec 

m-y 

r  v  =  0.091  sec 

m-x 

and  =  26.586  rad/sec/volt,  for  all  stages. 


4.3.4.  Computer  Simulation  Of  Uncoupled  Linear  Models 


In  this  section,  the  computer  simulation  model  of  the  linear  uncoupled  robots  under  digital 
computer  control  are  derived.  The  discrete  control  signal  u(kT)  is  fed  to  the  motor  by  a 
digital-to-gnalog  converter  (DAC).  Thus,  the  motor  input  is  held  constant  over  each  sampling 
period  T  (Figures  4-6  and  4-7). 

em(to).  Wm(t0) 


Saturation 

Nonlinearity 


Figure  4-6:  Rotational  Plant  Block  Diagram 


Saturation 

Nonlinearity 


Figure  4-7:  Translational  Plant  Block  Diagram 


Saturation  in  the  DAC  (and  equivalently  current  limiting  to  the  motor  amplifier)  is  included  in 

the  simulation.  Over  the  interval  [  kT  <  t  <  (k  +  1)T  ],  the  motor  voltage  is 
u(kT)  for  lu(kt)l<Vsa, 

ea(t)=  for  u(kT)>Vsal  (4.9) 

-VMt  for  u(kT)<-Vsa| 


In  the  model,  V  .  is  set  at  the  maximum  motor  driving  voltage  (40  Volts), 
sat 


A  discrete  model  of  the  analog  process,  which  specifies  the  output  at  the  sampling  instants, 
is  appropriate  for  the  computer  simulation.  However,  it  is  important  to  be  able  to  examine  the 
output  between  sampling  intervals.  Thus,  the  analog  output,  0m( t),  over  the  interval 


[t0<t<t0  +  T],  to  a  constant  step  input  of  magnitude  V  (volts)  applied  at  t  ,  with  initial 
conditions  0m(tQ)  and  <*>m(t0).  is  derived.  The  motor  speed,  wm(t),  is  also  derived  so  that  the 
initial  speed  can  be  determined  at  the  start  of  each  sampling  period. 


The  transfer  function  in  (4.1)  represents  the  differential  equation 

r  (9  +<9  =Ke  (t) 
mm  m  m  a 

which  can  be  written  as  two  scaler  state  space  equations: 


6  = « 
m  m 


w  =(-1/r  )cj  +  (K  /r  )e  (t) 

m  nr  m  m  m  a 


(4.10) 

(4.11) 


where  the  input  eg(t)  =  V  for  to<t<to  +  T.  The  solution  to  the  system  in  (4.10)  and  (4.1 1)  is 
0  (t)  -  0m(U  +  Tm[«(t)-KmVj  (  1  -e  (l  to)/Tm)  +  K  V(t-t  )  for  t  <t<t  +  T 

m  m  o  ml  o  m  J f  m  o  o —  — 

and 

«m(l)  =  w(n(t)e(,',o)/Tm  +  KmV(  1-e'(*V/Tm) 

m  '  m'  o'  m  '  / 


If  the  motor  displacements  for  the  turntable,  X  stage,  and  V  stage  are  0mfj,  6  n  x,  and  0m  , 
respectively,  then 


X(t)  = - 0  (t)  inches 

2wp  mx 

1 

Y(t)  = - 0  (t)  inches 

2wp  m-y 

360.0 

0(t)  = - 0  fl(t)  degrees 

2l7T|  m'e 


4.4.  Plant  Modeling  for  2  DOF  Robot  Arm 

A  2  DOF  articulated  robot  arm  is  modeled  in  this  section.  The  arm  is  actuated  by  DC  servo 
motors  which  are  mechanically  coupled  to  the  link  joints  by  gearing  mechanisms.  (Figure  4-8) 
The  motor  for  link  2  is  mounted  at  the  rear  of  link  1  to  act  as  a  counterweight.  The  motor  for 
link  1  is  mounted  in  the  stationary  base. 


The  following  assumptions  are  made  to  derive  the  dynamic  model: 


Motor  for  Link  1 


Link  2 


Link  1 


Motor  for  Link  2 


Stationary  Base 


Figu  re  4-8:  2  DOF  Robot  Arm 

1 .  Links  1  and  2  are  modeled  as  thin  rigid  rods  of  lengths  r1  and  r2,  and  mass  mM 

and  mr2,  respectively;  ~  f 

2.  Any  payload  (including  a  camera)  is  modeled  as  a  point  mass,  m  2,  at  the  end  of 

link  2;  p 

3.  The  gear  box  coupling  for  link  2  is  modeled  as  a  point  mass,  m  at  the  end  of 

link  1;  p 

4.  The  counter-weight  (motor)  is  modeled  as  a  thin  rigid  rod,  of  length  r/k5  and 
mass  m  ,  at  the  rear  of  link  1 ; 

5.  There  are  no  off-sets  between  links; 

6.  All  gearing  is  ideal;  and 

7.  All  bearing  friction  is  negligible 

This  simplified  geometric  arm  model  facilitates  development  of  the  mathematical  model.  The 


The  counter-weight  length  is  defined  as  a  fraction  (1/K)  of  the  link  length  to  facilitate  derivation  of  the  dynamic 
equations  in  Appendix  B. 
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mode!  results  in  a  nonlinear  dynamic  system  which  exhibits  the  Coriolis,  centrifugal, 
gravitational,  and  inertial  coupling  torque  components  present  in  a  real  arm. 

4.4.1.  Kinematic  Equations 

The  arm  kinematic  equations  describe  the  motion  of  the  arm  in  Cartesian  space.  These 
equations  are  required  to  derive  the  arm  dynamics  and  image  transforms. 

A  coordinate  frame  diagram  of  the  arm  is  shown  in  Figure  4-9. 


Figure  4-9:  Arm  Coordinate  Frames 

Each  link,  including  the  base,  is  assigned  a  Cartesian  coordinate  frame  in  accordance  with 
the  Denavit-Hartenberg  convention  [Denavit  55].  The  coordinate  frame  associated  with  the 
reference  base,  [Xo,Yo,Zo],  serves  as  the  world  coordinate  frame.  The  frames  associated 
with  links  1  and  2  are  [X ,,¥,,2,]  and  [X2,Y2,Z2]. 

The  position  of  the  end  of  link  2  with  respect  to  the  world  frame  is  derived.  Let 
T.  =  homogeneous  transformation  describing  the  position 
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of  link  j  relative  to  the  world  frame,  and 

A;  =  homogeneous  transform  describing  the  position 
of  link  i  relative  to  link  i- 1  (where  link  0 
corresponds  to  the  reference  base). 


Thus, 


T  =  A,  A„...A. 

I  1  2  i 


For  the  two  DOF  arm, 


Ci  Si  0  riCi 

•S,  C,  0  r,Sf 
0  0  10 


C2  s2  0  r2C2 
A  g  =  Sg  C2  0  ^^2 
0  0  10 


where 


C.icos  (0.) 
S  =sin(0 ) 


Thus, 


(4.12) 


VA1 


C12  -S12  0  + 

VA1A2=St2  C12  0  r2S12  +  riS1 
0  0  1  0 


where 

C  =cos(0j  +  0.) 

S.j=sin(0.  +  0.) 

From  (4.12),  the  rrobot  positions  are 

X(t)  =  r2cos( 0t(t)  +  02(t)  )  +  r1cos0,(t) 
Y(t)  =  r2sin(01(t)  +  02(t)  J  +  ^sinfl^t) 

4.4.2.  Rigid  Body  Dynamics 


The  dynamic  equations  for  the  manipulator  are  developed,  in  Appendix  B,  using  the 
Lagrangian  formulation.  The  dynamic  models  for  the  two  revolute  joint  angles  0  are: 


(4.13) 


F1  =  Dt1^1  +  D12^2  +  D1 12^1^2  +  °122^2  +  D| 

F2  =  D,20,  +  D2202+D2n012  +d2 


(4.14) 

(4.15) 


-  *.  . 


f  t  ft  C-f-j*' 


where 


D,1  =  +  3^  +  mP’  +  ™  2  "  f' +  2rir2°2 l-f  +  mp2]  +  h  mp2]  r 

°I2  =  [mp2  +  ^]r2  +  r.r2C2[^+mp2] 

D112  =  -2rir2S2C!y+mp2] 

Dl22  =  rir2^2t--^2+  fTlp2-l 

D1  =  9  {  +  mp,  r,S,  +  [^S,2  +  riS,  ]  mr2  +  [r2S,2  +  r,S1  ]  mp2) 

°22  =[mp2  +  ^]r2 

D2!  1  =  rir2S2l-*"J2  +  mp2l 

D2  =  9r2S|2[[mp2  +  ^2~\ 

In  the  simulations,  the  gravity  torques,  D  and  D  ,  are  set  equal  to  zero. 


4.4.3.  Actuator  Dynamics 

The  link  torques  F,  and  F2  are  related  to  the  actuator  dynamics.  The  DC  motor  actuator  for 
the  ith  link  is  modeled  Figure  4-10. 


The  motor  torque  F  (  driving  link  i  is 


Fmi  -  =  L*mi  +  «n/mi  +  FA 

so  that  the  link  torque  F.  is 

Fi  =  ^(V  Vmi"  VnAi 


In  terms  of  the  link  displacement,  0.  =  0m/Vj' l'nk  torques  are 


F1=^U'(ea1T'rKb17'l2d1)a1,»12<?rlm1T»1^1 


(4.16) 


F2-^12'(ea27f2'Kbl7J2  ^J'a2^2  ^2'lm27?2  ^2 
Ha2 


(4.17) 


fO  fO 


ei.F, 


applied  control  signal  voltage 

unity  gain,  high  B.W.,  high  impedance  amplifier 

armature-winding  resistance 

armature  current 

back  E.M.F. 

angular  displacement  of  motor  shaft 
motor  torque 

armature  moment  of  inertia 

motor  viscous- friction  coefficient 

gear  ratio  N2/N( 

angular  displacement  of  link 

link  torque 

back  emf  constant 

motor  torque  constant 

Figure  4-10:  Robot  Actuator 


(volts) 

(ohms) 
(amps) 
(volts) 
(radians) 
(oz-in) 
(oz-in -sec2) 
(oz- in/rad/sec) 
(rev.  motor/rev.  turntable) 
(radians) 
(oz-in) 
(volts/rad/sec) 
(oz-in/amp) 


4.4.4.  Robot  Arm  Parameters 

In  this  section  a  manipulator  is  designed  for  the  model  structure  assumptions  specified  at 
the  outset  of  this  section.  The  mechanical  structure  is  assumed  to  have  the  following 
dimensions  and  weights; 

•  Maximum  payload  =  5  lbs.  ( 1  pound  of  which  is  the  camera) 

•  Link  lengths  r1  =  r2  =  1 7  in. 

•  Weight  of  link  1=7  lbs 

•  Weight  of  link  2  =  5  lbs 

•  Weight  of  gear-box  =  1  lb 

The  motors  and  gearing  are  selected  according  to  the  design  criteria: 

1.  Maximum  individual  link  speed  =  180  deg/sec  =  n  rad/sec 


2.  Maximum  individual  link  acceleration  =  1800  deg/sec2  =  10w  rad/sec2 

3.  Minimize  motor  energy  dissipation  while  keeping  the  gear  ratio  tj  small  enough  to 
reduce  gearing  backlash  and  friction. 


The  design  steps  are  outlined  below,  and  are  performed  for  each  individual  link,  under  the 
assumption  of  negligible  dynamic  coupling: 

STEP  1 :  Find  the  torque  (T^  )  required  to  achieve  the  maximum  specified  acceleration 

for  the  maximum  payload  at  the  maximum  link  inertia. 

STEP  2:  Find  the  maximum  torque  (Tma)(  )  required  to  hold  the  maximum  payload  against 
gravity. 


STEP  3:  Specify  motor  torque  requirements: 

1 .  The  peak  motor  torque,  T  .  =  (T  „  +  T  J/ri 

^  ’  peak  '  max*acc  max-g'  1 

2.  The  continuous  motor  stall  torque,  T  .  „  =  T  /v 

stall  max-g  1 

3.  The  peak  motor  speed  as  180*t)  deg/sec  =  30»tj  RPM 


STEP  4:  To  select  the  gearing,  choose  a  motor  (which  is  anticipated  to  satisfy  the  torque 
requirements  in  Step  3)  and  set 


^ii-nominal^m^ 


1/2 


where  Dij  nomjna(  is  chosen  as  a  nominal  value  of  the  link  inertia.  This  selection  for  jj  minimizes 
the  energy  dissipation  in  the  motor  and  maximizes  the  power  transfer  to  the  link.  For  this 
design,  choose  Djj  nomjna|  as  the  average  of  DjS  evaluated  at  the  maximum  five  pound  load  and 
no  load  operating  points.  To  evaluate  D1V  it  is  assumed  that  02  =  45(deg).  If  ijj  is  too  large 
(e.g.,i}>35)),  then  it  can  be  reduced  by  a  third  with  only  a  17%  increase  in  motor 
dissipation  [Electro  Craft  80].  Finally,  check  if  this  motor/gear  combination  satisfy  the  motor 
torque  requirements  in  Step  3. 


While  ther  are  numerous  motor/gear  combinations  which  will  satisfy  the  stated 
requirements,  the  following  additional  guidelines  are  used: 

1.  Maintain  tj<35  to  minimize  backlash,  friction,  weight,  and  cost:  and 

2.  If  possible,  allow  for  a  safety  margin  in  the  peak  and  stall  torque  ratings. 


Before  proceeding,  the  weights  are  converted  to  mass  according  to 
m  =  W/g 

The  units  for  g  and  W  are  selected  so  that  the  torques  are  specified  in  units  of  oz-in.  Thus, 
W{lbs)16(oz/lb) 

m  - - s - 

32(ft/sec2)12(in/ft) 


and 


W(lbs) 

24 


(oz/in/sec2) 


mr1  =0.29167 

mr2  =  0.20833 

m  .  =0.04167 
pi 


mp2  =  0.20833  (with  5  lb.  load) 
mp2  =  0.04!67  (with  1  lb.  load) 
mc  =  (to  be  determined  in  motor  selection  process) 


The  motor  selection  process  proceeds  backward  from  the  last  link. 


4.4.4. 1 .  Motor  2  Selection 
Step  1: 

D22  =  (mf2/3  +  mp2)r22 

D22-max  =  80-27oz-in-s«c2 

and 

T  =  D  0 
max-acc  22- max  u  max 

=  2522  oz-in 


Step  2: 

VD2  =  9r2S12(mp2  +  'V2> 

Tmax-g  =  2040  ozin  (with  a  5  Lb-  ,oad) 

Step  3: 

Tpeak/T>2  =  (2522  +  2040)/i|2-  4562/t?2  oz'in 

Tstall/T»2  =  2040/',f2  OZ'in 

7j  X  peak  motor  speed  =  tj2  *30  RPM 


Step  4:  Before  calculating  the  gear  ratio,  it  is  noted  that  the  effective  inertia  of  motor  2 
includes  the  armature  inertia  and  the  non-negligible  inertia  of  the  motor-to-gear  shaft 
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coupling.  This  shaft  is  modeled  as  a  17x3/8  in  round  steel  shaft.  Assuming  the  density  of 
steel  is  .28  Ib/in3,  the  shaft  weighs  .52  lbs  (which  is  assumed  to  be  included  in  the  net  weight 
of  link  2).  Thus, 

2 

’shaft  =  •5mshaftrshaft 

=  0.00038  oz-in-sec2 

To  meet  these  requirements,  ELECTRO-CRAFT  660-E  motor  is  selected  (Table  4-4). 

I  =  0.032  oz-in-sec2 

m 

«  =  0.0065  oz- in/rad/sec 
111 

R  =  2.15  B 
a 

K.  =  0.24  volts/rad/sec 

b 

K{  =  34.19  oz-in/amp 
Volts(max)  =  43 

Weight  =  6  lb.  2  oz.  (mc  =  0.2552  oz/in/sec2) 
length  =  5.5 

Table  4-4:  D.  C.  Motor  E-660 

This  motor  is  over-rated  for  the  peak  and  stall  torque  requirements,  but  a  smaller  motor  would 
require  a  significantly  larger  gear  ratio.  This  motor  is  chosen  to  achieve  a  reasonable  gear 
ratio,  while  its  added  weight  serves  as  a  counter-weight. 


The  link  inertia  with  no  load  is 
D22  =  20.07  oz-in-sec2 
and,  the  nominal  link  inertia  is 

(80.27  +  20.07J/2  =  50.1 7  oz-in-sec2 
The  nominal  gear- ratio  is 

t)2  =  (50. 1 7/. 03238) 1 /2  =  39.36 
and,  reduced  by  33%  is 
ti2  =  26 

4. 4. 4. 2.  Motor  1  Selection 


Step  1: 


D11  "  +  "»0l  +  mr2  +  mp2l  rf  +  2r.r2C2[^  +  "\J  +  ' +  mp2]  r2 


3  3k  01 
=  424  oz-in-sec2 


and 
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T  =  D,.  e  =  13322  ozin 

nux-acc  1 1-max  max 


Step  2: 


Tg  max  =  °,  =  9  {  [^  +  r,S,  +  +  r,S,  ]  mf2  +  [r2S,2  +  r.S,  ]  mp2} 


=  2723  oz-in 


Step  3: 

T  k  =  (13322 +  2723)/tJj  =  16045/ij, 

Ts«all  =  2723/ tJi 

Step  4: 

D1 1 -full  load  =  371  oz  in  sec32  <at  6  =  45  de9) 
Dt  1-no-load  =  166  ozin  SeC  {3t  0  =  45  de9) 


^nominal  =  <371  +  166>/2  =  269 

To  satisfy  these  requirements,  Electro-Craft  motor  E-701  is  selected  (Table  4-5). 

=  40.77  oz-in/amp 
Kb  =  .288  v/rad/sec 
Ra  =  2.6Q 
|m  =  0.1  oz-in-sec2 
«  =  0.048  oz-in/rad/sec 

m 

Volts(max)  =  46.8 

Table  4-5:  D.C.  Motor  E-701 


The  nominal  gear  ratio  is 
7},  =  (269/0.1) 1/2  =  51 .86 
and  reducing  by  33% 

^=35 

4.4.5.  Computer  Simulation  of  the  Robot  Arm 

The  differential  equations  (4. 1 4),(4.1 5),(4. 1 6),  and  (4.17)  define  a  fourth  order  coupled 
nonlinear  dynamic  system.  The  simulation  uses  the  IMSL  [IMSL  82]  Fortran  library  routine 
"DVERK"  to  numerically  solve  these  equations.  The  routine  is  a  sixth  order  Runge-Kutta 
algorithm  with  an  automatic  variable  integration  step  size. 

To  implement  the  algorithm,  the  dynamic  equations  are  represented  as  the  system  of  first- 
order  differential  equations  &  =  F;XJ,  where  Ar  =  [X1,X2,X3,X4]  are  the  state -variables.  DVERK 


■  a  .  •" 
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produces  estimates  of  Xj(tQ  +  Tjnc)  from  X{tQ).  In  the  simulation,  T.  is  set  to  T/nk  (nk  =  integer) 
to  exhibit  the  output  between  sampling  instants.  (The  integration  step-size  is  typically  much 
smaller  than  T|nc  and  is  automatically  adjustea  in  DVERK  to  minimize  the  estimation  error.) 

The  system  is  represented  in  state- variable  form  according  to  the  following  assignments: 

Xi  =  *i 

V'i 

V*2 

Thus, 


From  (4. 1 4),(4. 1 5),(4. 16),  and  (4.17) 

F1  =  D11X2  +  DI2X4+  D112X2X4  +  °122X4  +D1 

(4.18) 

F2  =  D,2X2  +  D22X4  +  D2.,X22  +  D2 

(4.19) 

F1  =  j7L~(ealT,rKbl7,1  X2^'aiTfl  X2**ml1,1  X2 

(4.20) 

K3’ 

F2  =  ^2(ea27l2  Kb2TJ2  X4^a2T,2  X4  lm2I}2  X4 

Ra2 

To  simplify  the  notation  let 

Q=r^“(®aiTf,  ^bi^i  X2)'a1412x2 

Hal 

and 

Z=rIi(®a2T,2'Kb2,f2  X4^'a27f22X4 

Ha2 

(4.21) 

where  ea  is  defined  in  (4.9).  Equations  (4.18),(4.19),(4.20),  and  (4.21)  become 

0=  *  'm,0*2  + ' 0,2*4  *  D„2*2*4  *  °,22X42  * 

Z*^22+Im2,,2  ^X4  +  l-,12X2  +  D211X2a  +  ^2 

Upon  solving  (4.22)  and  (4.23)  for  X2  and  X4,  the  stats  model  is: 


x,  =  x2 


x  - - -2s _ 

2  D  D  -D  2 
ie2e  12 


X3  =  X4 


X  - - —IS _ 

4  D  D  -D  2 

la2e  12 


(0-^L[zD2„X/-D2]0tt2X!X,0|22X4!0,} 
^  2e 

{z-^-[Q-o1Hx,8.D„a: <2X4-0,  ]■ o,,  ,x2*( o2} 

le 


(4.22) 

(4.23) 

(4.24) 

(4.25) 

(4.26) 

(4.27) 


where 


^le  “  ®t1  f  'ml1*!, 
°2e  =  °22  +  lm2,,2 
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During  hierarchical  control  either  links  1  or  2  can  be  braked.  These  cases  are  simulated  as 
follows: 


Case  1:  link  1  •  free  and  link  2  -  braked 


Equations  (4.18)  and  (4.20)  reduce  to 

VD,i+D, 


and 


F.-O-'mll.X 


so  that 


VX2 

x2^ 

2  °ie 

Vo 


x4*o 


Case  2:  link  1  -braked  and  link  2-free 


The  state-space  model  becomes 


v° 

V° 


The  computer  simulation  of  the  robot  arm  is  depicted  in  Figure  4-11. 


4.4.5. 1 .  Linear  Model 

During  initial  analysis  of  IBVS,  it  is  useful  to  model  the  arm  dynamics  as  an  uncoupled  linear 
system.  The  MRAC  controller's  ability  to  adjust  to  the  nonlinear  feature  gains  can  then  be 
evaluated  independent  of  the  nonlinear  dynamics.  An  uncoupled  linear  model  is  obtained  by 
assuming  that  all  the  coupling  terms  in  (4.14)  and  (4.15)  are  zero,  while  D..  is  evaluated  at  the 
operating  point.  (Dj  is  set  to  zero  since  gravity  is  not  modeled  in  these  simulations.) 


Suturjt  ion 


Figu  re  4- 1 1 :  Arm  Simulation  Block  Diagram 


Equations  (4.14), (4.15), (4.16),  and  (4.17)  reduce  to 

— i-(e  ti.  K.  7i  20 )  «  ri.2( j.  l  tj20=D0. 
p  '  ai  *i  bi  *i  r  mi  *i  i  mt  *i  t  h  i 

ai 


To  apply  the  linear  model  routines  developed  in  Section  4.3,  the  displacement  of  the  ilh  link, 

0jt  is  converted  to  a  motor  displacement  according  to,  and  0.  =  <?mj/T}. 

K.  ....  .. 

d  '  ai  *i  bi  *i  mi'  mi  'i  mi  mi  *i  rrn  n  mi  fi 
Wai 

The  corresponding  transfer  function  G  .(s),  from  the  applied  voltage  to  the  motor  shaft,  is 

K  " 

G  (S)  = - !UL_ 


mi' 


S(TmjS+1) 


where 


no 


4.5.  Modeling  Picture  Taking  and  Feature  Measurement 

in  this  section,  a  model  for  the  vision  system  described  in  Chapter  2  is  developed.  The 
transduction  process  (or  camera  and  image)  are  modeled  by  defining  image  transformations 
which  map  3-dimensional  object  coordinates  into  2-dimensional  image  points.  The  image 
intensities  are  assumed  to  be  binary,  and  image  point  coordinates  are  considered  to  be 
analog.  Image  feature  measurements  are  then  derived  from  the  image  point  coordinates.  The 
dynamics  of  the  transduction-to-feature  extraction  process  are  modeled  by  a  unit  delay  (i.e., 
nd  =  1  Section  3.3).  Various  time  delay  durations,  and  thus  sampling  periods  T,  are  simulated 
to  evaluate  the  effect  of  different  sampling-to-bandwidth  ratios.  A  measurement  noise  model 
(which  can  represent  transducer  quantization,  thresholding  effects,  and  lighting  noise)  is  also 
included. 


4.5.1 .  Image  Transformations 


The  camera  is  modeled  by  defining  a  transformation  which  maps  3  dimensional  object 
coordinates  into  2-dimensional  image  coordinates.  To  perform  this  transformation,  the 
camera  is  modeled  as  a  pinhole  lens,  with  an  image  plane  lying  at  a  distance  F  behind'  the 
lens  [Duda  73].  To  produce  a  non-inverted  image,  a  frontal  image  plane,  defined  by  [xcam.Z 
cam] is  assumed  t0  lie  at  a  distance  F  in  front  of  the  lens.  Thus,  [YcamJ  defines  the  optical  axis. 
This  camera  model  defines  the  perspective  transformation, 
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For  a  three-dimensional  object  point  (Xre|.Yre,,Zre))  measured  in  the  camera  coordinate  frame, 
the  corresponding  two-dimensional  image  or  picture  coordinate  (Xpjc,Ypjc)  is  specified  by  the 
homogeneous  perspective  transformation 
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Zrel 
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■ 

L 1  +  Yre./FJ 

Thus, 

X  3  - ~r-Sl - 

p,c  1  +  Y  _,/F 


and 


Ill 


(4.28) 


The  picture  points  are  normalized  to  the  dimensions  of  the  image  plane  so  that  the  features 
(derived  in  Section  4.5. 2.1)  are  dimensionless.  If  it  is  assumed  that  the  image  plane  (e.n  .  film, 
or  ccd  chip)  is  a  square  with  sides  of  length  equal  to  ’ret’6,  then  (4.28)  becomes 


The  focal  length,  F,  is  adjusted  so  that  the  largest  object  dimension  occupies  approximately 
25  to  33  percent  of  the  image  in  the  desired  view.  This  allows  for  a  margin  of  motion  of  the 
object  within  the  image  during  control. 


4.5.2.  Feature  Transformations 

Image  features  are  derived  from  the  idealized  non-distorted  two-dimensional  image  points 
(X  .  ,Z  ).  Image  distortions,  which  can  arise  from  the  numerous  factors  noted  in  Chapter  2, 

pic  pic 

are  difficult  to  model  and  vary  widely  with  lighting  conditions,  transducer  resolution  and 
linearity.  These  distortions  are  manifested  as  measurement  noise  and  limit  the  system 
accuracy,  dynamic  performance,  and  relative  stability.  To  evaluate  characteristics  of  IBVS 
control  under  noisy  conditions,  simulations  are  performed  in  Chapter  5  using  additive 
measurement  noise.  Noise  modeling  is  included  in  Section  4.5.2.3. 

4.5.2.1 .  Two-Dimensional  Line 

Two  features  are  derived  for  the  line  in  space  simulations  (for  Configurations  1  and  2). 
According  to  the  configuration  specification,  the  perceived  angle  of  the  line’s  image  remains 
constant,  and  only  two  features  can  be  obtained  from  such  a  binary  image: 

1 .  length  of  line  •  length;  and 

2.  X  center-of  gravity  of  line  •  Xcog 

These  features  are  derived  as  follows.  The  position  of  the  center  of  the  line  (Figure  4-1) 
relative  to  the  image  plane  is  (Xret.Y  ,).  The  end  points  of  the  line  (Figure  4-12),  measured  in 
the  camera  coordinate  system,  are 

a 

In  the  simulations,  tet  =  0.25  inches  (eg.,  approximating  the  dimension  of  a  typical  CCD  device  such  as  the 
Fairchild  CCD211). 
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<X,.Y,) .  ([X  e,  ,  .5  l„COS«  J.[Y,a  ♦  .5  l„sin«  j) 
(X2.Y2)=([X|e|..5loC0S*e,].[Yrd..5losin9ie,]) 

From  (4.28)  and  (4.29),  the  perceived  line  length  is 


I  =  X  -X 
ength  pic-1  pic-2 

4FI  (F  +  V  JcosO  X  .sin0  . 

=  — 2» - [e! — s-121 — ^ (dimensionless) 

rel  4(F  +  Yrel>  ('oSin<?re.) 
and,  the  perceived  center-of-gravity  is 

v  _  *pic-i  +  XdIc-.2 


F  4X  .(F  -t-  Y  )-l  2sin0  .COS0  . 

=  — • — — - !-g-  ° - — — 5 — —  (dimensionless) 

ret  4(FrYrel)-!.(losin8re|)2 


(4.29) 


(4.30) 


(4.31) 


It  remains  to  calculate  Xre|,  Y  (,  and  0fe|  as  a  function  of  the  object  and  robot  positions.  For 
the  Cartesian  system  (Figure  4-1): 


Xre.<l>  =  XobrX(t) 

Yre,(t);Yob,Y<t> 
^rel  =  ^obj 


(4.32) 


For  the  articulated  arm  (Figure  4-2),  coordinate  transformations  are  used.  The  basic 


transformation  is 


where 


"^"^canwel  Yobj 


T2  =  end  of  arm  relative  to  world 

Tcam  =  image  plane  relative  to  end  of  arm 

Tre,  =  line  relative  to  image  plane 

T  . .  =  line  relative  to  world 
obi 


(4.33) 


From  (4.33) 


T  =  T1  t’T 
rel  cam  2  obj 


(4.34) 


Tcam  is  sPecified  by  the  assumptions  of  Section  4.2.2  to  be  a  rotation  about  the  Z  axis  of  the 
arm.  Thus, 


K 
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and 
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1 
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0 

0 
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Since  T„  is  defined  in  (4. 1 2), 


^12  '^12  ®  '(r)C2  +  r2) 


^12  ^12 


0 

0 


1 

0 


f1S2 

0 
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Since 


obi 


CO  .  -SO  . .  0  X  .. 

oh|  ob|  obi 

SO  ..  CO  0  Y  ' 

nb|  obi  obi 

0  0  10 


0  0  0 
then  according  to  (4.34) 


1 


Tre.= 


C<?ob,  ^obi  0  Xob,  (»-2C,2  +  r,C,) 
S*obi  C*obj  0  Yobi(r2S,2  +  riS,) 

0  0  1  o 

0  0  0  1 


Hence 


xrel(t)  =  Xobr[r2C0S(d1(t)  +  6  2<l>)  +  ',0080,(1)] 
Yrel(t)  =  Yobj'Cr2Sin(^1^)  +  *2(t))  +  r1sin5,(t)] 


^rel  =  ^obj 


(4.36) 


During  the  simulation  experiments,  three  conditions  are  evaluated  to  assure  that  the  derived 
features  are  valid.  The  conditions  determine  if  the  line  is: 

1.  In  front  of  the  camera  (or  image  plane); 

2.  In  the  field  of  view;  and 

3.  Not  hidden  (it  is  assumed  that  only  one  side  of  the  line  is  visible) 

If  any  of  these  conditions  are  violated,  then  the  experiment  is  terminated.  The  conditions  are 
evaluated  as  follows: 

Condition  1:  true  if  (Y1  or  Y2)  >  0;  else  false 


Condition  2:  true  if  IX  .lor  IX  ,.l  <  ret/2;  else  false 

pic- 1  pic-.; 

Condition  3:  true  if  #re|  <  arctan(Y(e|,X(gl);  else  false. 


4. 5. 2. 2.  Polyhedral  Objects 


(4.37) 


The  third  configuration  uses  convex  polyhedral  objects  for  the  control  tasks.  Each  object  is 
defined  by: 

1.  Labeling  it’s  constituent  vertices  and  identifying  their  coordinates  in  the  object 
coordinate  system;  and 

2.  Defining  the  object  planes  by  assigning  to  each  an  ordered  listing  of  the 
bounding  vertices. 

Three  objects  (a  cube,  pyramid,  and  wedge)  are  shown  in  Figure  4-13. 
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Figure  4-13:  Objects 

For  each  visible  object  plane,  two  features  are  obtained: 

1.  Perceived  area  of  the  i(h  plane  -  Afea  and 

2.  Perceived  X  center-of-gravity  (or  centroid)  of  the  i,h  plane  -  Xcog 

These  features  are  typical  of  those  extracted  from  binary  images,  in  commercially  available 
systems,  following  image  segmentation.  It  has  been  suggested  that  such  features,  which  are 
based  on  integral  or  summation  operations,  are  preferable  for  visual  servoing  applications, 
since  these  operations  attenuate  signal  noise  [Hasegawa  77].  Feature  selections,  which  can 
include  both  linear  and  nonlinear  combinations  of  the  extracted  features,  are  presented  in 
Chapter  7. 


Each  object  vertex  is  transformed  into  the  image  plane  as  follows:  A  point  defined  in  the 
object  coordinate  system  is  defined  by 
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Vobj=(Xobj*Yobi-Zobi'1)T 

And  it's  position  relative  to  the  frontal  image  plane  is 


Vret  =  (XrerYre.'Zre.'1)1 


(4.38) 


Then, 


V  =T*T  *V 
rel  t  1  obj  obj 

•where  the  homogeneous  transformation  T  defines  the  position  of  the  turntable  coordinate 
system  [Xturn,Y|urn,Z(urn],  relative  to  the  camera’s  coordinate  system  [Xcam.Ycan).ZcaJ.  The 
position  of  the  object  coordinate  system,  relative  to  the  turntable  coordinate  system,  is 

defined  by  Tobj.  Then,  T(  is  found  by  the  following  transformation: 

X,x(t) 

T  =  TRANS[Y,R]*ROT[X,«]*TRANS  X,y(t)  *ROT[Z,0(t)] 

Z,-h 

C0(  t)  S0(t)  0  x(t) 

=  S0(t)Ca  C0(t)  -S a  y(t)Ca  +  hS«  +  R 
S0(t)Sa  C0(t)S0  C a  y(t)Sa-hCa 
0  0  0  1 

If  the  object  frame  is  referenced  to  the  turntable  by  translations  in  the  X-Y-Z  directions 
corresponding  to  (P  ,P  ,P  )  and  oriented  according  to  the  Euler  convention  by  (<p, £,</•)  [Paul 

x  y  z 

81],  then 


C<pCeC4'-S<pSif'  -C<pCeS'p-Sq>Ci//  C<pSe  Px 
Tob-  =  SrpCeCip  +  C<pS^  ■StpCeSip  +  C<pC*p  S«pSe  P 
-SeC9  SeS<p  Ce  P* 

0  0  0  1* 

Finally,  V.e|  is  mapped  into  homogeneous  image  points  V  via  the  perspective  transformation, 
T  ,  according  to 


V-T  «T  »T  *V  — (V  V  V  S1T 
p  t  'obj  obj-'x’  y’  i'0* 


Hence, 

XPic  =  VS 
ZpiC  =  VS 

In  addition,  the  image  points  are  normalized  to  the  image  plane  size  (’ret’)  according  to 
X  .  <-X  .  /ret 

pic  DIC 
Zpic*“Zpic/ret 

Once  the  object  planes  are  mapped  into  object  image  planes  (by  finding  the  perceived 
vertices),  the  features  are  calculated.  The  image  plane  is  either  a  quadrilateral  or  triangle 
(Figure  4-14).  The  area  and  centroid  of  the  triangle  are  [Kindle  50] 
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Figure  4-14:  Image  Planes 

Area  =  '°-5(XiZ2  X2Z3+ X3Zl'X1Z3'X3Z2'X2Zl^ 

and 

_xjL>x2lxa 

cog  3 

The  area  of  the  quadrilateral  is  given  by 

Area  =  _0-5(X,Z2  +  X2Z3  +  X3Z4  +  X4Z1X1Z4X4Z3'X3Z2X2Z|) 

The  centroid  of  the  quadrilateral  can  be  determined  by  the  following  algorithm  [Marks  78]: 
The  quadrilateral  is  divided  into  two  sets  of  triangles  by  means  of  the  diagonals 
(Figure  4-14).  Find  the  centroid  of  the  four  triangles  belonging  to  the  same  set.  The 
intersection  of  these  lines  will  be  the  centroid  of  area.  Thus,  in  the  figure,  O,  Or 
02,  and  03  are,  respectively,  the  centroids  of  the  triangles  ABD,  ABC,  BDC,  and 
ACD.  The  intersection  of  0103  with  002  give  the  centroid. 
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In  accordance  with  the  line  example,  the  three  conditions  in  (4.37)  are  evaluated  for  each 
plane,  during  the  simulations,  to  assure  that  the  features  are  valid.  If  the  ith  plane  image 
features  are  sought,  the  conditions  are: 

Condition  1:  true  if  Vfe(  (for  all  vertices  comprising  the  ilh 
plane)  >  0  ;  else  false 

Condition  2:  true  if  IX  .  I  and  IV  I  (for  all  vertices 

pic  .L-  pic 

comprising  the  i  plane)  <  ret/2  ;  else  false 

Condition  3:  Each  object  plane  is  defined 

by  assigning  to  it  an  ordered  listing  of  the  bounding 
vertices.  Specifically,  the  vertex  labels  are  specified 
in  a  clockwise  ordering.  In  3-dimensional  space,  when 
two  connected  edge  vectors  of  a  plane,  whose  head  and 
tails  are  defined  in  a  clockwise  fashion,  are  cross 
multiplied  then  the  magnitude  of  the  resultant  vector 
has  a  positive  sense  according  to  the  "right-hand" 
rule.  If  the  plane  containing  these  edge  vectors  are 
rotated  or  translated  out  of  the  field  of  view,  then  the 
perceived  sense  of  the  original  defining  vertices 
would  be  counter-clockwise  and  the  vector  cross-product 
of  the  perceived  edge-  vectors  would  have  a  negative 
sense. 


4. 5. 2. 3.  Noise  Signal  Modeling 

Single  DOF  simulation  studies  (in  Chapter  5)  include  additive  feature  measurement  noise. 
The  noise  signal  is  modeled  as  zero-mean  white-noise  having  a  uniform  random  distribution 
(Figure  4-15),  which  is  scaled  by  the  factor  to  produce  a  specified  signal-to-noise  ratio 
(SNR). 


P(») 


Figu  re  4- 1 5:  Scaled  Distribution 

The  noise  signal,  n(k),  is  generated  according  to 
n(k)  =  a[RAN(k)-.5] 

where  RAN(k)  is  the  TOPS-20  Fortran  library  routine  which  generates  independent  samples 


from  the  uniform  random  distribution.  To  determine  a.  the  SNR  is  specified  with  respect  to  the 
average  power  of  a  step  function  signal: 
s2  =  (reference  signal  •  initial  value)2 

Thus, 


SNR  =  10  log  — j 

CTx 

where  a  2  is  the  variance  of  the  noise,  and  for  a  uniform  distribution 


(4.39) 


tr  2  =  a2/12 


Hence,  for  a  specified  SNR,  the  scale-factor  is 


a2  =  12*10t'°9  s  snr/1°) 


4.6.  Summary 


In  this  chapter,  visual  ser/oing  tasks  are  defined  for  evaluation  in  subsequent  chapters. 
Mathematical  models  of  robot  kinematics  and  dynamics,  picture  taking,  and  feature 
transformation,  which  are  required  for  task  simulations,  are  also  developed.  The  tasks  vary 
from  control  of  1  DOF  to  3  DOF  systems,  and  are  summarized  in  Table  4-6.  The  task 
configurations  are  selected  to  facilitate  the  IBVS  evaulation  of  progressively  complex 
dynamics,  kinematics,  and  image  feature  transformations,  and  will  help  to  understand  the 
relative  contributions  of  each  to  the  control  issues.  While  these  configurations  have  limited 
degrees-of-freedom,  which  may  be  useful  for  some  practical  tasks,  their  models  may  be 
considered  representative  of  more  general  configurations.  It  is  believed  that  the  results 
obtained  with  these  examples  may  then  be  applied  to  more  general  cases. 


Name  Robot  Type/Task 


Model  Derivation 


Conf  i  gur.it  ion  1 
( F igrue  41) 


•2  DOF  with  I  in«.ir  uncoupled 
dynamics  and  kinematics 
(evaluated  in  Chapter  6).  or 
1  DOF  by  constraining  motion 
to  one  axis 

(evaluated  in  Chapter  5) 

•Control  position  of  a  mobile 
camera  (fixed  to  robot  end-effector) 
relative  to  a  fixed  line  in  space 


Configuration  Definition:  Section  4.2.1 
Robot  Model:  Section  4.3.2 

Feature  Measurement:  Sections  4.5.1. 

4.5.2.  and  4.5.2. I 


•2  DOF  with  nonlinear  coupled 
dynamics  and  kinematics 
(evaluated  in  Chapter  6) 

♦Control  position  of  a  mobile 
camera  (fixed  to  robot  end -ef fector ) 
relative  to  a  fixed  line  in  space 


Configuration  Definition:  Section  4.2.2 
Robot  Model:  Section  4.4 

Feature  Measurement:  Sections  4.5  1. 

4.5.2.  and  4.5.2.  1 


Conf i gurat ion  3 
(F  igrue  4-3) 


•3  DOr  with  lineal'  uncoupled 
dynamics,  and  a  combination  of  2 
planar  Cartesian  with  1  rotational 
DOF 

♦Control  position  of  a  mobile 
polyhedral  object  (fixed  to  robot) 
relative  to  a  stationary  camera 


Configuration  Definition:  Section  4.2.3 
Robot  Model:  Section  4.3 

Feature  Measurement:  Sections  4.5.1. 

4.5.2.  and  4. 5. 2. 2 


Table  4-6:  Configuration  Summary 
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Chapter  5 

Evaluation:  One  DOF  Systems 

5.1 .  Overview 

The  purpose  of  this  chapter  is  to  evaluate  SP  MRAC  adaptive  control  of  one  DOF  linear  and 
nonlinear  systems.  The  evaluation  analyzes  examples  which  are  relevant  to  the  two  and  three 
DOF  visual  servoing  tasks  evaluated  in  Chapters  6  and  7.  While  single  DOF  IBVS  systems  may 
be  useful  for  specific  robotic  applications,  these  results  serve  to  help  understand  MRAC 
control  of  nonlinear  I8VS  systems,  and  as  a  guideline  for  implementing  and  evaluating  higher 
degree-of-freedom  systems  required  for  visual  servo  control. 

The  chapter  is  organized  as  follows.  In  Section  5.2,  MRAC  is  introduced  for  the  control  of  a 
simple  motor  positioning  system.  These  examples  are  then  used  for: 

•  Verification  of  the  basic  FP  and  SP  MRAC  algorithms  developed  by  Morris  and 
Neuman  [Neuman  80]  for  constant  linear  systems,  and 

•  Verification  of  the  algorithms  which  include  a  measurement  delay  (Section  3.3) 

In  Section  5.3,  IBVS  feature  feedback  is  introduced,  and 

•  Extends  SP  MRAC  control,  with  delay,  to  include  a  constant  feature  sensitivity 
gain,  and 

•  Demonstrates  the  superior  noise  filtering  performance  of  the  fixed  LMFC 
controller  relative  to  MRAC  control. 

In  Section  5.4,  control  is  extended  to  include  a  nonlinear  feature  sensitivity  gain,  and 

•  Demonstrates  the  stability  of  SP  MRAC  control  for  this  nonlinear  system, 

•  Evaluates  position  response  when  the  controlled  signal  is  nonlinearly  related  to 
position,  and 

•  Includes  a  performance  comparison  of  MRAC  vs.  Fixed  LMFC  control,  which 
demonstrates  superiority  of  the  adaptive  approach  at  smaller  system  bandwidth’s 
and  the  superiority  of  the  fixed  approach  at  smaller  sampling-to-bandwidth  ratios. 
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5.2.  Baseline  MRAC 

The  purpose  of  this  section  is  to  verify  the  SP  and  FP  MRAC  algorithms  (derived  in  Chapter 
3),  for  systems  with  and  without  measurement  delay,  by  simulating  a  simple  motor  positioning 
control  task.  The  position  of  the  Y-translational  stage  (Figure  4-7),  which  is  measured 
directly,  is  the  variable  under  control.  The  examples  included  in  this  section  focus  on  the  SP 
MRAC,  since  this  controller  is  used  for  all  subsequent  IBVS  control.  While  the  FP  MRAC 
exhibits  faster  tracking  of  the  reference  signal  (without  a  delay),  the  SP  MRAC  is  selected  for 
IBVS  control  for  the  following  reasons: 

1.  Relative  to  the  FP  reference  model,  a  SP  reference  model  reduces  the  high 
frequency  gain  of  the  feedback  path,  reducing  the  noise  amplification. 

2.  In  contrast  to  the  FP  MRAC's  deadbeat  response  characteristic  to  recovery  from 
control  signal  saturation,  the  SP  MRAC  is  not  susceptible  to  deadbeat. 

3.  While  the  relative  stability  of  the  FP  MRAC  is  susceptible  to  control  signal 
saturation,  the  SP  MRAC  is  not  as  sensitive. 

4.  The  SP  MRAC  requires  a  smaller  control  penalty  with  measurement  delay. 

The  Y-translational  stage  dynamics  represent  a  constant  linear  SISO  system.  For  this  type  of 
system,  the  MRAC  algorithms  are  verified  by  showing  that  the  identifier  (3.4),  (3.5),  and 
(3.6)  can  perfectly  identify  the  plant  parameters,  and  by  showing  that: 

1 .  For  FP  MRAC,  without  measurement  delay,  there  is  perfect  modeling  following, 
when  e|D(k)  =  0,  according  to  (A. 14).  It  is  also  verified  that  relative  stability  with 
this  type  of  control  is  susceptible  to  control  signal  saturation. 

2.  For  SP  MRAC,  without  measurement  delay,  the  FP  error  is  a  filtered  identification 
error  according  to  (A.23).  Slower  tracking  of  the  reference  model  occurs  when 
identification  errors  exist.  The  SP  MRAC,  however,  is  not  as  sensitive  to 
saturation. 

3.  For  FP  MRAC,  with  measurement  delay,  a  control  penalty  is  required  for  absolute 
stability  according  to  the  root-locus  analysis  in  Figure  3-6. 

4.  For  SP  MRAC  (with  measurement  delay),  a  control  penalty  is  required  for  relative 
stability  according  to  the  root-locus  analysis  shown  in  Figure  3-8. 

The  discrete  open-loop  plant  model  is  calculated  below  for  comparison  with  the  parameters 

estimated  by  the  identifier.  The  open-loop  system  gain  (in  Figure  4-7)  is 

K  =  K  — *SF  =  K  =  (26.58)  (— )(100)  =  60.43 

m  2wp  V2tt7/V 


m 


when  the  scale  factor  SF  (Section  3.6.3)  is  set  equal  to  100.  The  analog  model  (4.5)  of  the 
process  under  control  becomes 


G(S)  =  - 


60.43 


s(.072s  +  1) 

and  the  corresponding  discrete  model  of  is 

1  b.z^+b-z'2 

W  (z1)=-J - ; — ^ — 5 

p  1-atz'  -a2z  2 


where 


a.  =  1  +  e'T/r  ;  r'=  .072 
a2  =  -e'T/T 

b  =-(Kr2)(1-I/r-e  T/T)  ;  K  =  60.43 
b2=  (Kr3)(le'T/T-T/re'T/f) 


Since  the  plant  dynamics  has  real  poles  (i.e.,  at  z  =  1  and  z  =  .632),  there  is  no  upper-bound 
constraint  on  the  sampling  period  selection.  The  sample  period  is  selected  as  T  *  0.033  (sec) 
to  correspond  to  the  period  used  in  initial  IBVS  simulations.  For  this  sampling  period,  the 
actual  parameter  vector  (A.  10)  becomes 
£  =  [.3945  .3387  1.63235  -.63235]T 


5.2.1 .  MRAC  Without  Measurement  Delay 

In  the  first  set  of  experiments,  the  MRAC  algorithm  for  systems  without  delay  (i.e.,  nd  =  0)  is 
verified.  The  controller  parameters,  listed  below,  are  empirically  selected  (according  to  the 
guidelines  in  Section  3.6)  and  the  controller  algorithm  is  implemented  according  to: 

1.  Control  signal  (A.  12) . 

2.  The  identifier,  (A. 15), (A. 16),  and  (A. 17),  uses  second  order-estimators  (i.e.,  for  a 
second-order  plant)  with  a  weighting  factor  A  =  0.85  .  In  the  first  trial  of  each 
experiment,  the  estimated  parameter  vector  is  initialized  to 

£(0)  =  [10  10  1.1  -.1]T 

which  represents  large  gains  and  fast  poles  so  that  the  system  is  not  forced  to 
start  out  too  fast,  causing  overshoot.  The  adaptive  gain  matrix  is  initialized  to 
P(0)  =  DIAG[1010  80  80] 
and  reset  with  a  threshold  of 

P.hresh=250° 

A 

Each  experiment  is  repealed,  for  a  second  trial,  with  £(0)  and  P(0)  initialized  to 
the  steady- state  values  of  £  and  P  learned  in  the  first  trial. 


3.  SP  reference  model  (A. 22).  The  reference  model  bandwidth,  or  the  closed-loop 
system  bandwidth,  is  specified  by  (3.29).  With  a  performance  margin  PM  =10 
(Section  3.6. 2. 2),  the  bandwidth  is 

it  i 

u  = - =  9.52  sec 

n  10T 

This  bandwidth  corresponds  to  a  rise-time  tf  =  .7  (sec)  according  to  (3.26),  and  a 
sampling -to-bandwidth  ratio  of  fs/fBW  =  20  according  to  (3.28).  The  open-loop 
reference  model  time  constant,  rmode|  =  0.048,  is  chosen  to  achieve  this 
bandwidth  according  to  the  methods  outlined  in  Section  3.6.1.  Discrete  model 
parameters  are  then  generated  according  to  the  method  outlined  in  Appendix  A. 

The  unit-step  response  of  the  SP  MRAC  is  displayed  in  Figure  5-1.  In  the  first  trial,  the 
identification  error  goes  to  zero  in  .4  sec.  Unlike  an  FP  MRAC,  the  FP  error  does  not 
immediately  go  to  zero  due  to  the  model  filtering  effect  predicted  by  (A. 23).  In  the  second 
trial,  ejd(k)~0  for  all  k,  which  results  in  perfect  model  following.  The  estimated  parameter 
vector,  at  the  end  of  the  first  trial,  does  not  correspond  to  the  actual  vector.  This  can  be 
expected  since  the  control  signal  may  not  be  sufficiently  rich  in  frequency  content  [Yuan  77] 
to  identify  a  physical  model.  However,  by  at  the  end  of  the  second  trial  there  is  perfect 
parameter  identification.  While  saturation  does  not  occur  in  this  experiment,  the  examples  in 
subsequent  sections,  which  exhibit  saturation,  do  not  display  the  deadbeat  response 
associated  with  an  FP  MRAC. 

The  simulations  verify  the  the  baseline  MRAC  algorithms.  The  next  section  includes  an 
evaluation  of  these  algorithms  extended  to  include  a  measurement  delay. 

5.2.2.  MRAC  With  Measurement  Delay 

In  this  section,  a  measurement  delay  is  included  by  setting  nd  =  1.  The  controller  is 
implemented  according  to: 

1.  Control  signal  (3.10). 

2.  The  identifier,  (3.4), (3.5),  and  (3.6),  is  initialized  as  in  Section  5.2.1. 

3.  SP  reference  models  (A. 22).  The  reference  model  bandwidth  remains  at  wn  =  9.52 
sec'1.  The  open-loop  reference  model  time  constant  becomes  'rrnocjei  =  0.036 
according  to  the  methods  outlined  in  Section  3.6.1.  And,  the  discrete  model 
parameters  are  generated  according  to  the  methods  outlined  in  Section  3.3.3. 

4.  Control  penalty  (3.12).  (The  controller  pole  bound,  y,  will  be  specified  for  each 
experiment). 
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Figure  5-1:  SP  MRAC  Without  Measurement  Delay 


The  root-locus  in  Figure  3-8  predicts  that  the  SP  MRAC  should  be  stable  for  all  7r(k).  In  the 
simulation  displayed  in  Figure  5-2,  the  controller  pole  bound  is  initially  set  to  y  =  1  .  After  the 
initial  learning  trial,  the  system  responds  with  a  rise-time  .7  (sec),  with  a  decaying  small 
amplitude  oscillation  superimposed.  By  the  end  of  this  trial  there  is  perfect  identification. 

A  A 

Since  Ibj/b^y  ,  then  w(k)  =  1,  and  the  dominant  closed-loop  poles  at  (,099±.931j)  predict 
the  observed  asymptotic  oscillation  frequency.  Since  the  angle  between  the  poles  and  the  real 
axis  is 

0  =  w.T 

d 

then  these  poles  correspond  to  a  frequency  of 

tan-1(.93i/.099) 

f  = - ^ - -  =  7  Hz. 

2ttT 

The  magnitude  of  oscillation  can  be  attenuated  by  decreasing  the  pole  bound  to  y  =  0.3  ,  thus 
increasing  the  control  penalty.  The  oscillation  is  suppressed  at  the  expense  of  a  larger 
rise-time,  tf~0.8  (sec).  While  increasing  control  penalty  improves  stability,  it  produces  results 
equivalent  to  reducing  the  system  bandwidth,  and  leads  to  larger  tracking  errors. 

These  experiments  summarized  here  verify  the  basic  SP  and  FP  MRAC  algorithms  by 
demonstrating  adaptive  control  of  a  simple  positioning  system.  While  fixed  controllers  can  be 
designed  for  this  system,  based  on  a'priori  knowledge  of  the  plaid,  the  adaptive  controller 
required  minimal  a'priori  information.  Next,  the  SP  MRAC  is  extended  to  control  a  linear 
image-based  system.  The  SP  MRAC  is  selected  for  IBVS  control  since  it  is  less  susceptible  to 
saturation  and  noise,  arid  requires  smaller  control  penalties  required  for  measurement  delays. 


5.3.  Linear  IBVS 

One  DOF  IBVS  systems  are  simulated  using  Configuration  1  (Figure  4-1),  and  restricting 
relative  motion  to  either  the  X  or  Y  directions.  When  motion  is  in  the  X  direction,  either  the 
length  (4.30)  or  center-of-gravity  (4.31)  features  can  be  used  for  control.  For  each  case,  the 
feature  sensitivity  scaler,  J)eat,  becomes  either 


9  X  F 

_ CQfl- _ 


4X,Dl(F  +  YrJ 


ret  4(F  +  Y  .)  (I  sin#  ,)^ 


rel  Hf.VJ'iWJ 


If  6r.  =  0  or  7r  radians,  then  91  ../9X  .  =0  and  (5.2)  degenerates  to  zero.  The  center-of- 


gravity  feature  is  thus  selected  for  one  DOF  control  of  the  X-axis.  Since  this  system  has  linear 
kinematics  and  the  camera  is  mounted  with  the  image  plane's  axis  parallel  to  the  translational 
stage  axis,  the  arm  jacobian  scaler  is  Jarm  =  I .  The  sensitivity  (2.7)  becomes 
3  ~  3arm  3feat  ~  3feat 

Since  Jfeat  is  independent  of  Xre|,  the  system  under  control  is  linear.  The  discrete  open-loop 
model  parameters  are  obtained  from  Section  5.2  by  including  the  feature  sensitivity  J  in  the 
gain  K: 

1 

K  =  J  K  - SF 

m  2irp 

The  value  selected  for  SF  depends  upon  the  sampling-period  and  the  feature  sensitivity 
magnitudes  (since  the  discrete  gains  decrease  with  sampling-period  and  sensitivity).  For  all 
IBVS  examples  in  Chapters  5  and  6,  the  scale  factors  are  empirically  determined  to  be: 

SFT=  033  =  2500 

SFT=  0,3  =  5000 

SFr=  003=  10000  (5'3> 

The  SP  MRAC  control  parameters  are  initialized  according  to  Table  5-1 . 


Diag  [10  10  80  80] 

2500 

[±10  tlO  1.1  -0. 1]  T 

Diag  [1111] 

100 

[±1  tl  1.5  -0.5]  T 

Diag  [ . 1  .1  .01  .01] 

10 

[±1  -tl  1.5  -0.5]  T 

Table  5-1:  Identifier  Parameter  Initialization  (1  DOF) 


In  Table  5-1,  the  initial  estimated  parameter  vectors,  £(0),  represent  large  gains  and  fast 
poles  so  that  the  system  is  not  forced  to  start  out  too  fast.  In  practice,  more  accurate  initial 
guesses  can  be  made  to  achieve  faster  convergence  of  the  identifier.  As  T  decreases,  the 
gain  terms  {bj}  decrease,  and  open-loop  poles  move  closer  to  the  unit  circle.  The  initial 
estimated  parameter  vector  is  varied  with  T  to  reflect  these  changes.  In  the  initial 

A  A 

experiments,  the  the  estimated  gain  terms  {b  1  (0),b2(0)} ,  are  initialized  to  the  incorrect  sign  to 
demonstrate  MRAC  sign  sensitivity.  This  sensitivity  becomes  an  important  aspect  of 
independent  MRAC  control  of  coupled  systems,  and  is  discussed  in  Chapter  6.  When 
y  =  0.85,  the  linear  IBVS  systems  still  exhibit  a  decaying  very  low  amplitude  and  frequency 


oscillation.  By  further  reducing  the  controller  pole  bound  to  y  =  0.65,  a  smoother  response  is 
obtained. 


In  the  simulations  presented  in  this  section,  the  sampling  period  is  set  to  T  =  0.033  (sec),  and 
the  specified  close-loop  bandwidth  remains  wn  =  9.53  sec'1,  so  that  the  sampling-to- 
bandwidth  ratio  is  fg/fBW  =  20.  Two  control  tasks,  labeled  Tasks  A  and  B  in  Figure  5-3, 
represent  time-invariant  linear  IBVS  systems. 


Position 


Figure  5-3:  Linear  Time- Invariant  Tasks 

For  each  task,  a  picture  is  taken  when  the  camera  is  in  the  desired  position  and  the  extracted 
X„og  feature  value  is  used  as  the  reference  step-input.  The  simulation  of  Task  A  is  displayed 
in  Figure  5-4.  The  FP  output  and  identification  errors  are  expressed  in  terms  of  the  scaled 
feature.  For  this  configuration,  the  actual  parameter  vector  is 
£  =  [-3.8984  -3.4550  1 .6958  -  .6958] 

A  A 

when  T  =  0.033,  rx  =  0.091,  and  = -0.4849  .  In  the  initial  trial,  b1(0)  =  b2(0)=  +10,  and  the 
system-starts  off  in  the  wrong  direction  due  to  the  incorrect  sign  of  the  initial  estimated  gain 
terms.  As  the  system  learns,  the  estimated  gain  becomes  negative  and  the  table  reverses 
direction.  After  this  learning  trial,  the  system  starts  out  in  the  correct  direction  in  the  second 
trial.  By  the  end  of  the  second  trial,  the  estimated  parameter  vector  equals  the  actual 
parameter  vector.  The  system  rise-time  has  increased  to  tr=  .9  (sec),  and  there  is  a  larger  FP 
output  error  due  to  the  increased  control  penalty.  The  sensitivity  of  the  MRAC  controller  to 


the  sign  of  the  gain  term  is  an  important  aspect  of  adaptive  control.  For  SISO  control  of 
multi-DOF  systems,  it  is  shown  in  Chapter  6  that  sign  sensitivity  can  degrade  path  trajectory 
response. 


Task  B  is  specified  to  force  control  signal  saturation  resulting  from  the  longer  distance  that 
must  be  traveled.  The  position  response  and  control  signals  are  displayed  in  Figure  5-5. 


Figure  5-5:  IBVS  with  Control  Signal  Saturation 
The  camera  focal  length,  F,  is  reduced  so  that  the  line  does  not  go  gut  of  field  of  view  (OFV). 
Therefore,  the  feature  sensitivity  is  reduced  to  3Xcog/3Xre)  =  -0.0773,  thus  decreasing  the 
plant  gain.  The  first  trial  uses  parameter  and  adaptive  gain  values  initialized  to  the  values 
learned  from  the  Task  A  simulation.  While  there  is  control  signal  saturation,  no  deadbeat 
response  results.  This  experiment  demonstrates  the  ability  for  the  same  adaptive  controller  to 
operate  equally  well  with  different  plant  sensitivities. 
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5.3.1 .  Noise  Performance:  Fixed  vs.  Adaptive  Control 

A  potential  drawback  of  any  identification  and  feedback  control  strategy  is  susceptibility  to 
measurement  noise.  The  control  algorithm  and  measurement  system  must  be  designed  to 
reduce  noise.  Fixed  controllers  are  inherently  less  susceptible  since  they  do  not  include 
on-line  identification.  In  computerized  vision  measurement  systems,  image  pre-processing 
becomes  the  primary  mechanism  for  noise  reduction.  Post- processing,  or  filtering,  of  the 
estimated  positions  or  features  may  be  required  for  further  noise  reduction.  The  MRAC 
algorithm  provides  inherent  means  for  minimizing  noise  effects.  Morris  [Morris  79]  has 
demonstrated  that: 

•  The  SP  reference  model  has  superior  noise  performance,  relative  to  the  FP 
reference  model. 

•  The  control  penalty  enhancement  attenuates  the  gain  of  the  feedback  path,  and 
thus  noise  amplification. 

•  The  gain  of  the  feedback  path  can  be  further  attenuated  by  reducing  the  closed- 
loop  system  bandwidth  (i.e.,  by  specifying  a  slower  reference  model). 

•  The  least-squares  weighting  factor  A  provides  a  noise  filtering  effect  for  the 
estimation  process. 

In  the  next  series  of  experiments,  the  relative  noise  performance  of  the  LMFC  and  MRAC  are 
compared  for  two  values  of  A.  Fixed  controllers  are  implemented  by  fixing  the  gains  of  the 
MRAC  controller,  and  turning-off  the  adjustment  mechanism.  Fixed  gain  values  are  derived 
by  fixing  the  estimated  parameter  vector  to  the  actual  parameter  values. 

Task  A  (Figure  5-3)  is  simulated  with  added  measurement  noise.  The  focal  length  has  been 
reduced,  to  F  =  25  (mm),  to  keep  the  line  in  the  field  of  view  when  smaller  SNRs  are  used. 
Noise  performance  is  evaluated  at  five  noise  levels,  including  infinity  (i.e.,  no  noise),  80,  60, 
44,  and  30  dB.  For  example,  the  SNR  =  44  dB  corresponds  to  quantization  noise  due  to  pixel 
resolution  for  a  512  X  512  CCD  array  without  further  pre-processing.  For  each  SNR,  an 
experiment  consists  of  five  trials.  The  first  two  correspond  to  successive  adaptive  trials  with 
A  =  0.85,  the  next  two  correspond  to  A  =  1 ,  and  the  final  trial  corresponds  to  a  fixed  LMFC  with 
the  estimated  parameter  vector  set  equal  to  the  actual  parameters: 

£(k)*-/l  =  [-2.88453  -2.55645  1.69584  -,69584]T 
Each  trial  uses  the  same  noise  sequence.  The  simulation  results  are  shown  in  Figures  5-6  and 


5-7  for  noise  levels  of  44  and  30  db,  respectively.  The  X-stage  starts  off  in  tho  wrong 

A 

direction,  for  SNR  =  30  db  and  A  =  0.85,  since  the  estimated  gain  b(  term  is  positive  at  the  end 
of  the  first  trial.  At  20  dB,  the  line  would  not  stay  in  the  field  of  view. 
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Figuro  5-6:  MRAC  vs.  LMFC  Noise  Performance  (SNR  -  44  db) 


The  relative  noise  performance  is  evaluated  by  defining  the  error  index  (E)  according  to 
E  =  log£^j32R,(k)  J  forkT>2sec 

which  is  a  measure  of  steady-state  error7.  The  error-index  vs.  SNR  is  plotted  in  Figure  5-8  for 
the  second  learning  trial  at  each  A.  The  fixed  controller,  which  exhibits  a  performance  which 
is  linearly  related  to  the  noise  level,  performs  better  than  the  MRAC  controller  when  A  =  0.85  . 
When  A  =  1,  the  LMFC  and  MRAC  have  similar  performance.  However,  A  is  set  to  less  than 
one  wfien  controlling  the  nonlinear  systems  described  in  the  following  sections.  While  ether 
approaches  have  been  noted  for  noise  reduction,  such  as  larger  control  penalties  and  smaller 
bandwidths,  it  appears  that  independent  noise  filters,  operating  on  the  measurement  signals, 
would  be  required  to  achieve  suitable  responses,  at  lower  SNRs. 


Without  noise,  tho  lull-parallel  error  is  essentially  zero  after  2  seconds  The  error  index,  E.  is  thus  calculated  for 
kT>2sec.,  so  that  that  E,  calculated  without  noise,  approaches  minus  infinity,  and  is  the  reference  level 
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5.4.  Nonlinear  IBVS 


When  motion  is  restricted  to  the  Y  direction,  the  sensitivity  J.  become 
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When  X|e)  =  0  and  0re|  =  0,  then  3Xcog/3Yre|  =0  and  (5.4)  is  degenerate.  Therefore,  lenglh  is 
chosen  as  the  feature  for  depth  control.  Since  Jf  is  a  function  of  Y  (,  the  system  under 
control  has  linear  dynamics  and  nonlinear  gains.  For  example,  in  Figure  5-9,  the  camera 
moves  along  a  trajectory  specified  by  Task  The  corresponding  feature  transformation,  (5.4), 
and  the  corresponding  feature  sensitivity,  (3leng,h/3Y)-  are  plotted  in  Figure  5-10  for  various 
values  of  line  angle  (0obj).  These  plots  demonstrate  that  the  gains  can  change  rapidly  and 
dramatically,  especially  as  the  camera  approaches  tho  object.  For  these  systems,  stable 
adaptive  performance  is  expected  (if  the  identifier  is  fast  enough  to  track  the  gain  changes), 
with  responses  similar  to  the  linear  cases,  with  the  following  exceptions: 

•  Since  the  variable  under  control,  feature  length,  is  nonlinearly  related  to  the  plant 
variable,  Y,  the  time  response  of  Y  does  not  match  the  time  response  of  the 
feature. 

•  Since  the  process  under  control  is  nonlinear,  there  is  no  reason  to  expect  that  the 
estimated  plant  parameters  will  converge  to  the  to  the  physical  parameters. 

•  Larger  values  of  sampling -to-bandwidth  ratios  and  smaller  values  of  \  will  be 
required  to  track  time-varying  gain  terms. 

To  study  the  behavior  of  these  systems,  the  simulations  of  Tasks  C  and  D  (Figure  5-9)  are 
repeated  for  various  values  of  0Qbj.  Tasks  are  simulated  in  both  the  approach  direction  (i.e., 
camera  approaches  the  line  as  indicated  by  the  arrow  in  Figure  5-9),  and  in  the  backing  away 
trajectory.  Task  C  represents  a  relatively  small  motion  task,  with  a  sensitivity  that  changes  by  a 
factor  of  2.2  over  the  trajectory.  Task  D  represents  a  large  motion  trajectory,  with  sensitivity 
that  changes  by  a  factor  of  90.  With  respect  to  the  identifier  requirements,  both  cases  can  be 
considered  as  extreme  examples  because  the  gains  change  very  rapidly  as  the  camera  comes 


Figure  5-9:  1  DOF  Nonlinear  Tasks 

in  dose  proximity  with  the  line.  If  the  tasks  had  been  specified  so  that  the  camera  were 
further  away  from  the  line,  but  with  a  larger  focal  length  (F)  to  increase  the  magnification,  the 
sensitivity  changes  would  not  be  as  large.  The  relationship  between  the  sensitivity  and  the 
focal  length  and  relative  distance  (V  ,)  is  given  by  (5.4).  For  example,  with  =  0,  (5.4)  is 

-spate  =  -Ja. - - — -  (5.6) 

3Yre.  ret  ( F  +  Yre|)2 

As  F  and  Y  .  increase,  the  sensitivity  decreases. 


For  the  initial  studies,  the  SP  MRAC  is  implemented  according  to  the  aforementioned  linear 
examples  (Section  5.2.2),  with  A  =  0.85,  un  =  9,5  sec'1,  and  T  =  0.033  .  The  following 
observations  emerge  from  specific  simulation  studies.  In  all  simulations,  the  performance  is 
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Figu  re  5-10:  T ask  D  Feature  Characteristics 

stable  and  the  steady-state  identification  error  goes  to  zero.  Transient  identification  errors 
always  remain.  Both  type  0  and  type  I  estimated  plants  result.  Since  type  0  plants  may  be 
estimated,  a  signal  bias  (Section  0.4. 1 )  must  be  used  to  the  reduce  steady-state  errors  to  zero. 
When  type  I  plants  are  estimated,  the  estimated  poles  do  not  match  the  actual  poles.  In  many 
cases,  non-minimum  phase  zeros  are  estimated.  Thus,  a  control  penalty  is  required, 
irrespective  of  the  penalty  required  for  feedback  delay. 

As  the  least  squares  weighting  factor  X  is  reduced,  the  controller  becomes  too  sensitive  to 
tracking  rapidly  changing  gains,  especially  for  tasks  where  the  camera  approaches  the  object 
over  large  distances.  The  controller  tries  to  decelerate  too  quickly  as  it  estimates  rapidly 
increasing  gains.  This  can  cause  temporary  reversal  of  table  direction.  Conversely,  as  X 
increases,  smoother  responses  result,  but  with  possioly  small  underdamped  target  overshoots 
and  longer  settling  times.  The  author  has  found  that  the  value  X  =  .85  provides  an  acceptable 
tradeoff.  In  applications  where  small  overshoots  can  be  tolerated,  larger  values  can  be  used. 
However,  the  most  effective  way  to  track  rapidly  changing  parameters  is  to  increase  the 
sampling-to-bandwidth  ratio. 


When  the  feature  signal  is  nonlinearly  related  to  position  (e.g.,  Tasks  C  or  D),  the  feature 
and  position  time  responses  will  be  different.  Since  control  of  position  is  the  goal,  it  is 
important  to  understand  the  relationship  between  position  and  feature  time-domain 
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responses.  To  investigate  this  relationship,  the  the  time  response  of  forward  and  reverse 
trajectories  are  analyzed  below,  since  the  observed  responses  exhibit  the  following 
interesting  relationships: 

1.  When  approaching  the  object,  feature  rise-times  are  smaller  than  feature  rise- 
times  when  backing  away  from  the  object. 

2.  When  approaching  the  object,  position  rise-times  are  less  than  the  corresponding 
feature  rise-times. 

3.  When  backing  away  from  the  object,  position  rise-times  are  greater  than  the 
corresponding  feature  rise-times. 

4.  Consequently  the  position  rise-times  associated  with  approach  trajectories  are 
smaller  than  the  backing  away  position  rise-times. 

The  explanation  for  these  observations  are  discussed  in  conjunction  with  the  specific 
examples  which  follow. 


First,  Task  C  is  simulated  with  0Qb.  =  45  degrees.  The  response,  which  is  displayed  in  Figure 
5-11,  includes  the  initial  and  subsequent  trials  in  the  forward  direction.  The  third  trial,  which  is 
in  the  opposite  direction,  uses  the  estimated  parameter  vector  and  adaptive  gain  matrix 
initialized  to  values  learned  from  the  second  trial.  The  rise  times  for  this  example  are 
Approach  Feature  Rise-Time:  tf  -  0.9  sec  (2nd  trial) 

Backaway  Feature  Rise-Time:  tf  =  1 .5  sec 
Approach  Position  Rise-Time:  tr  *  0.83  sec  (2nd  trial) 

Backaway  Position  Rise-Time:  tf  =  1 .55  sec 

Since  the  feature  signal  is  controlled  directly,  it  is  expected  that  the  approach  and  backing 
away  feature  rise-times  would  be  identical.  In  the  approach  trajectory,  the  feature  rise-time 
matches  that  of  the  linear  IBVS  example  Task  A,  however  in  backing  away,  the  rise-time  is 
much  larger.  The  explanation  for  these  results  requires  evaluation  of  the  gain  change  with 
time.  The  gain  change,  dJ/dt,  which  must  be  tracked  by  the  identifier,  is  evaluated  according 
to 


dJ  3J  .  dY 

”” dt __ 3Y*~dT  (5'7) 

where  3J/3Y  is  the  second  derivative  of  the  feature  transformation  (5.4).  The  gain  change 
(5.7)  for  Task  C  is  plotted  in  Figure  5-12  for  both  approach  and  reverse  trajectories.  The 
maximum  rate  of  gain  change  is  larger  when  backing  away  since  the  camera  starts  in  a  region 
where  3J/3Y  is  large,  and  during  the  initial  part  of  the  critically  damped  feature  response 
dY/dt  is  large.  Upon  approaching  the  line,  the  initial  3J/3Y  is  smaller,  and  during  the  final 
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Figure  5-12:  Sensitivity  Change  for  TASK  C 

part  of  the  trajectory  dY/dt  decreases.  Since  dJ/dt  is  larger  for  backing  away,  the  transient 
identification  error  is  larger  during  the  time  when  most  of  the  motion  occurs.  A  longer  time  is 
required  to  recover  from  identification  tracking  errors,  and  the  rise  time  increases. 


Analysis  of  the  relationship  between  feature  and  position  response  follow.  It  is  shown  below 
that  the  predicted  position  approach  times  (i.e.,  the  response  that  would  occur  for  perfect 
model  following,  or  epp(k)  =  0.0  for  all  k)  are  smaller  than  in  a  reverse  trajectory,  irrespective  of 
tracking  errors  due  to  sensitivity  changes.  While  this  analysis  provides  insight  into  IBVS,  it 
may  have  potential  implications  for  task  planning  to  minimize  cycle  times.  To  derive  the 
position  signal  as  a  function  of  the  feature  signal,  the  expression  for  the  length  feature 
(4.30)  is  solved  for  Yre(1  with  Xre|  =  0: 

'4  "><  U.h»Y-=l  4  Fl2-<4  F  SF  C°S  4  F>-(re'  'en0,h  *"  V  -  °  <5  8> 

For  each  sampling  period,  (5.8)  is  solved  for  Y  ,(k)  as  a  function  of  the  length  feature 
reference  model  signal  length.ref(k).  This  analysis  is  used  to  predict  the  position  response  if 
perfect  model  following  were  achieved.  (Actual  response  will  lag  behind  the  predicted 
response  due  to  control  penalties,  transient  identification  errors,  and  possibly  control  signal 
saturation).  From  (5.8)  for  Task  C,  the  predicted  position  rise-time  in  a  forward  trajectory  is 
0.69  seconds,  and  0.79  seconds  in  the  reverse  direction.  While  the  position  excursion  is  the 


141 


same,  the  sensitivity  change  differences  account  for  the  position  time  response  differences. 
These  predicted  results  are  explained  as  follows.  The  position  approach  rise  time  is  smaller 
than  the  corresponding  feature  rise  time  since  0Y/3I  |h  decreases  as  the  camera 
approaches  the  line.  Thus,  Y  can  be  close  to  it's  final  value,  while  the  length  feature  can 
change  by  larger  amounts  to  reach  to  it's  final  value.  Upon  backing  away  from  the  object, 
3Y/3I  (h  increases,  and  larger  changes  in  V  are  required  for  the  same  changes  in  the  length 
feature. 

The  effect  of  the  samplingto  bandwidth  ratio  f  /fQW  on  system  peiformonce  is  evaluated 
below.  Task  C  is  simulated  for  #ob|  varying  between  0  and  80  degrees,  using  a  samplingto- 
bandwidth  ratio  of  fs/fBW  =  20  (which  is  the  same  value  used  for  the  linear  IBVS  control  in 
Section  5.3).  The  simulations  all  exnibit  good  stability,  and  the  approach  position  rise  times 
remain  essentially  constant  (tr  =  .83  sec),  while  the  feature  rise-times  increase  with  increasing 
0  .  Motions  over  larger  distances,  and  thus  larger  gain  change,  require  larger  sampling  to- 

bandwidth  latios  to  achieve  equivalent  performance.  Performance  with  movement  over  a 
distance  of  18  inches,  represented  by  Task  D  (Figure  5  9),  is  displayed  in  Figure  5-13.  Rise- 
times  become  larger  because  of  control  signal  saturation.  The  oscillation  in  position  is  due  to 
larger  identification  errors,  as  a  result  of  larger  gain  changes  (Figure  5-10).  It  was  noted,  at 
the  outset  of  this  section,  that  decreasing  X  to  reduce  ejD  makes  the  system  too  sensitive  to 
the  rapidly  changing  gains.  An  effective  means  to  smooth  this  response  is  to  increase  the 
sampling-to-bandwidth  ratio.  For  example,  Figure  5-14  compares  the  response  for  three 
ratios,  which  are  varied  by  fixing  the  bandwidth  at  wn  =  9.5  sec'1,  and  changing  the  sampling 
period  according  to 

I  T  =  0.033  (fs/fGW  =  20) 

2.  T  =  0.013  (fg/fBW  *  50) 

3. T  =  0.003(fs/fBW  =  220) 

Overall  time  responses  are  comparable,  but  become  smoother  as  the  sampling-to-bandwidth 
ratio  increases.  At  the  highest  ratio,  the  identification  error  becomes  negligible.  In  the  next 
section,  it  is  shown  that  an  advantage  of  fixed  control  is  that  it  can  exhibit  superior  stability  at 
the  lower  ratios. 
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5.4.1 .  Adaptive  Control  Vs.  Fixed 

A  performance  comparison  of  adaptive  versus  fixed  control,  for  the  nonlinear  IBVS  system, 
is  presented  here.  Fixed  controllers  are  implemented  by  fixing  the  gains  of  the  MRAC 

A 

controller.  Fixed  gain  values  are  derived  from  the  estimated  parameter  values  ft  which  are 
identified  from  initial  adaptive  learning  trials.  The  studies  include  the  effects  of  varying  the 
range  of  motion  (and  thus  the  change  in  feature  gain),  bandwidth,  sampling-to-bandwidth 
ratio,  and  the  strategy  for  initial  mode!  parameter  estimation.  The  results  of  these  experiments 
suggests  the  following: 

•  For  a  specific  task,  and  at  a  sufficiently  large  bandwidth,  a  fixed  controller  can  be 
tuned  to  achieve  a  time  response  comparable  to  the  adaptive  controller.  As  the 
bandwidth  decreases,  fixed  control  performance  becomes  sluggish  relative  to  the 
adaptive  response. 

•  A  fixed  controller  tuned  for  a  specific  task  cannot  perform  as  well  for  another 
task.  Adaptive  controllers  can  perform  well  over  a  wide  range  of  tasks. 

•  Fixed  control  performance  is  smoother  than  adaptive  performance  (i.e.,  improved 
stability)  at  lower  samp'ing  to-bandwidth  ratios. 

F.ach  task  in  Figure  5-9,  including  tasks  at  intermediate  distances,  are  simulated  at  two 
reference  model  bandwidths;  wn  =  9.5  sec'1  and  un  =  4.4  sec'1.  In  both  cases,  the  sampling 
period  is  set  to  T  -0.033  (sec).  For  each  task,  at  each  bandwidth,  three  control  modes  are 
evaluated: 

1 .  Adaptive; 

A 

2.  Fixed  Control;  where  /[  is  fixed  to  the  values  learned  in  the  initial  adaptive  run; 
and 

A 

3.  Fixed  Control;  where  H  is  fixed  to  the  values  learned  from  adaptive  control  of  Task 
C.  • 

Task  performance,  which  is  summarized  in  Figures  5-15  and  5-16,  is  evaluated  by  the 
measured  position  rise-times.  At  the  higher  bandwidth,  the  adaptive  controller  and  the  fixed 

A 

controller  (with  ^learned  over  the  entire  trajectory)  have  coinparable  performance.  When  the 
fixed  system  is  tuned  for  a  high  gain  region  (i.e.,  Task  C),  it  performs  sluggishly  when  required 
to  track  gain  changes  in  lower  gain  regions.  Other  initial  learning  strategies  are  possible. 
However,  if  learning  is  biased  to  the  low  gain  region,  the  fixed  controller  becomes 
underdamped,  and  possibly  unstable,  as  it  approaches  the  target.  At  the  lower  bandwidth, 
fixed  control  performance  degrades,  relative  to  adaptive  control,  as  AJ  increases.  One  of  the 


explained  as  follows.  A  plant  KW(z  )  is  controlled  with  a  fixed  feedback  controller  (Figure 
5-17),  where  the  gain  term,  K,  varies  with  time.  The  feedback  compensators  are  C(z-1)  and 
HU'1). 


Figure  5-17:  Fixed  Feedback  Control 


The  closed-loop  transfer  function,  MU'1), 
C(z',)KW(z'1) 


MU'1)  = 


1+C(z'1)H(z',)KW(z'1) 


(5.9) 


is  the  same  form  as  the  LMFC  represented  in  Figure  3-7.  Over  the  frequency  range  of  interest, 
the  transfer  function  (5.9)  approaches 

1 


MU'1)- 


HU'1) 


(5.10) 


independent  of  the  plant  gain.  Therefore,  as  the  closed-loop  bandwidth  increases,  the  system 
becomes  less  sensitive  to  gain  variations  [Kuo  82]. 


At  the  higher  bandwidth  (Figure  5-15),  the  adaptive  response  degrades  as  AJ  and  the 
distance  traveled  increase,  as  a  result  of  control  signal  saturation  and  larger  transient 
identification  errors.  At  the  lower  bandwidth  (Figure  5-16),  the  adaptive  performance  remains 
essentially  constant  because  the  sampling-to-bandwidth  ratio  has  increased,  thus  reducing 
e,D(k),  and  because  saturation  is  reduced  for  the  smaller  required  velocities. 


At  the  outset  of  this  section,  it  is  noted  that,  for  small  excursions  and  relatively  small 
changes  in  J,  the  adaptive  response  is  smooth.  As  the  distance  traveled  increases,  and  AJ 


increases,  small  oscillations  result  unless  f«/fow  is  increased.  Fixed  control  results  in  a 


smooth  response  even  at  the  lower  fs/fBW  ratios.  For  example,  Figure  5-18  compares  the 


adaptive  versus  fixed  responses  for  Task  D,  with  T  =  0,033  sec.  and  u_  =  9.5  sec.  An 
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advantage  of  fixed  control  is  that  it  exhibits  superior  stability  at  lower  sampling-to-bandwidth 
ratios.  This  becomes  an  important  consideration  when  the  hardware  used  in  an  actual 
implementation  constrains  the  sampling  period. 
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IBVS:  TASK  D 

Figu  re  5- 1 8 :  Position  Response  of  Fixed  vs.  Adaptive  Control 

5.5.  Summary 

The  experiments  included  in  this  chapter  verify  the  simulation  of  the  lv!RAC  adaptive 
controllers  (Chapter  3),  for  SISO  systems.  The  SP  MRAC  is  selected  for  IBVS  control  since  it 
less  sensitive  to  control  signal  saturation,  has  improved  relative  stability  with  measurement 
delays,  and  has  superior  noise  performance. 

The  analysis  shows  that  for  IBVS  control,  the  system  under  control  exhibits  nonlinear  open- 
loop  gains  (or  sensitivities)  which  can  change  dramatically  over  the  task  trajectory.  A 
contribution  of  this  dissertation  is  that  MRAC  control  of  such  nonlinear  systems  (with 
measurement  delay)  produce  dynamically  stable  closed-loop  systems,  in  simulation,  without 
directly  interpreting  actual  positions.  Achievement  of  acceptable  stability  then  requires 
attention  to  the  following  details: 

1.  A  least  squares  weighting  factor  of  A  =  0.85  provides  a  suitable  trade-off  between 
tracking  feature  sensitivity  changes  and  over-reaction  to  those  changes.  Larger 
values  may  be  used  to  improve  noise  response. 

2.  The  sampling-to-bandwidth,  f  /f0w  ratio  must  be  selected  in  terms  of  the 

magnitude  and  slope  of  feature  sensitivity  change.  For  example,  as  the  required 

distance  of  travel  increases,  larger  sensitivity  changes  result,  which  require  larger 

ratios.  While  fg/fRW  —  20  is  suitable  for  a  small  motion  task  (e  g.,  with  a  1  inch 

travel,  where  the  sensitivity  changes  by  a  factor  of  2.2),  the  ratio  must  be 

increased  to  f  /L...  =  50  for  an  10  inch  excursion  task,  in  which  the  sensitivity 
s  uw 

changes  by  a  factor  of  90.  The  specific  sampling-to-bandwidth  ratios  also  depend 
on  the  features  used  for  control  and  relative  object  orientations. 


« 
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3.  A  controller  pole  bound  of  y  =  0.65  is  suitable  for  damping  out  oscillations  due  to 
measurement  delays.  The  control  penalty  is  also  required  for  stability  since  the 
identifier  can  estimate  non-minimum  phase  zeros  arising  from  the  nonlinear  gain 
terms. 

It  is  straightforward  to  design  a  fixed  control  law  for  a  linear  system  when  the  control 
engineer  has  detailed  knowledge  of  the  open-loop  system.  The  adaptive  controller  is 
implemented  with  minimal  a' priori  knowledge.  When  the  plant  is  nonlinear,  or  has  unknown 
parameters  (e.g.,  for  the  ’teach-by-showing'  strategy),  it  is  a  formedable  engineering  task  to 
design  a  suitable  fixed  control  law.  A  contribution  of  this  dissertation  is  to  evaluate  an 
approach  for  implementing  fixed  controllers  for  such  nonlinear  system.  The  steady-state 
estimated  plant  parameters,  derived  by  MRAC  control,  are  used  to  specify  a  fixed  LMFC 
controller.  Comparison  of  the  adaptive  and  fixed  control  approaches  suggests  the  following: 

1.  While  the  stabity  of  an  adaptive  controller  is  highly  susceptible  to  measurement 
noise,  especially  as  X  decreases,  a  fixed  controller  exhibits  superior  noise 
performance. 

2.  A  fixed  controller  exhibits  a  better  transient  response  than  the  MRAC  as  the 
sampling  to  bandwidth  ratio  decreases.  An  adaptive  approach  has  superior  time 
responses  at  lower  closed-loop  bandwidths. 

3.  A  fixed  controller  tuned  for  one  task  may  not  be  suitable  tor  another  task.  A  single 
adaptive  controller  can  be  suitable  for  many  tasks. 

The  following  chapter  extends  the  evaluation  of  adaptive  MRAC  and  fixed  LMFC  to  the 
independent  control  of  multiple  DOF  coupled  systems. 


! 
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Chapter  6 

Evaluation:  Two  DOF  Systems 

6.1 .  Overview 

This  chapter  includes  an  evaluation  for  IBVS  control  of  both  linear  and  nonlinear  kinematic 
two  DOF  configurations.  The  linear  kinematic  configuration  (Section  4.2.1)  is  modeled  with 
uncoupled  linear  robot  dynamics.  The  kinematic  structure  of  this  configuration  is  uncoupled 
in  a  Cartesian  frame,  permiting  the  camera  to  be  mounted  so  that  the  robot’s  degrees-of- 
freedom  coincide  with  the  camera's  reference  frame  (i.e.,  the  translational  X  and  Y  axes  are 
parallel  to  the  image  plane's  frame  of  reference).  The  articulated  arm  configuration  (Section 
4.2)  displays  nonlinear  and  coupled  kinematics  and  dynamics.  In  this  configuration,  the 
articulated  degrees  of  freedom  (i.e.,  0 ,  and  $2)  are  nonlinearly  related  to  the  camera’s  frame. 
Because  of  the  kinematic  and  dynamic  distinctions,  these  configurations  are  evaluated 
separately.  While  the  evaluations  focus  cn  image-based  control,  enhanced  SP  MRAC  joint- 
level  control  of  the  nonlinear  dynamic  robot  is  also  evaluated. 

The  chapter  is  divided  into  two  parts.  The  first  part  (Section  6.2)  includes  the  evaluation  of 
the  linear  kinematic  configuration.  In  the  second  part  (Section  6.4),  the  nonlinear  and  coupled 
kinematic  and  dynamic  configuration  is  evaluated,  along  with  the  enhanced  SP  MRAC  joint- 
level  controller.  (The  controller  parameters  are  summarized  in  Appendix  C.)  The  evaluations 
include  an  analysis  of  the  sensitivity  matricies,  and  compare  the  control  system  performance 
of  both  uncoupled  adaptive  and  fixed  controllers.  Potential  performance  advantages  of  a 
coupled  controller  are  suggested  by  isolating  specific  problems  associated  with  independent 
control. 

The  chapter  includes  detailed  explainations  and  interpretations  of  the  responses  observed 
in  the  simulations.  The  predominant  observations  are: 

•  Adaptive  IBVS  control  achieves  dynamically  stable  systems  for  both 
configurations.  The  system  performance  exhibits  good  transient  response,  and 
zero  steady-state  errors.  This  performance  is  achieved  without  explicit  a'priori 
knowledge  of  object  positions  or  robot  dynamics. 


•  For  these  two  degree-of-freedom  systems,  predicted  paths  (i.e.,  those  paths 
which  would  result  if  perfect  feature  reference  model  following  were  achieved) 
are  nearly  straight  line  trajectories.  It  is  encouraging,  for  future  implementations 
of  more  general  systems,  that  these  paths  do  not  exhibit  tortuous  trajectories. 

•  The  "teach-by-showing"  strategy  provides  a  convenient  means  for  specifying 
visual  servoing  tasks  when  stringent  path  control  is  not  required.  For  example,  it 
may  be  used  for  small  motion  precision  positioning  (e  g.,  in  precision  assembly, 
for  parts  aquistion  and  mating  alignment),  and  for  gross  positioning  when  smooth 
motion  is  acceptable  and  the  work  area  is  free  of  obstacles. 

•  Independent  controllers  produce  deviations  from  the  predicted  path.  Typical 
deviations  are  relatively  small,  with  the  exception  of  large  motion  nonlinear 
kinematic  tasks.  In  these  cases,  large  deviations  increase  the  risk  of  obstacle 
collisions,  and  make  it  difficult  to  keep  the  observed  object  in  the  field  of-view. 

Thus,  independent  controllers  may  not  be  suitable  for  such  tasks. 

•  Another  limitation  of  Ihe  independent  control  strategy  is  accentuated  by  the 
kinematic  coupling  of  the  nonlinear  kinematic  configuration.  In  this  configuration, 
there  is  a  requirement  for  a  feature/joint  reassignment  strategy.  In  contrast,  a 
fixed  feature/joint  assignment  is  suitable  for  the  linear  kinematic  configuration. 

•  The  enhanced  SP  MRAC  controller,  with  delay,  yields  stable  systems,  with  good 
transient  response,  when  used  as  a  joint-level  controller.  To  the  best  of  the 
author’s  knowledge,  this  is  the  first  simulation  demonstration  of  an  enhanced 
MRAC  (i.e.,  SP  MRAC,  with  control  penalty  and  measurement  delay)  for  robot 
control. 

•  An  MRAC  can  be  used  to  derive  a  fixed  LMFC  control  law.  ThE  fixed  controllers 
have  better  relative  stability  at  lower  sampling-to-bandwidth  ratios,  and  tend  to 
exhibit  faster  rise-times.  Relative  to  the  MRAC,  however,  they  produce  larger  path 
deviations,  and  are  only  suitable  for  small  motion  tasks  which  have  smaller 
sensitivity  changes.  A  single  fixed  controller  cannot  achieve  uniform  performance 
over  a  broad  range  of  tasks. 

6.2.  Linear  Kinematics  and  Dynamics 

This  section  includes  an  evaluation  for  the  linear  kinematic  and  dynamic  system 
represented  by  Configuration  1  (Figure  4-1).  The  adaptive  control  approach  is  demonstrated 
to  exhibit  dynamically  stable  responses,  and  zero  steady-state  feature  and  position  errors. 
Predicted  position  trajectories  (i.e.,  the  predicted  pathes  if  perfect  reference  modeling  were 
achieved)  are  smooth  and  approximate  straight-line  trajectories.  Actual  position  trajectories 
deviate  from  the  predicted  response,  however,  typical  deviations  are  relatively  small.  The 
MRAC  time  response  lags  the  specified  reference  model  rise-times.  The  fixed  LMFC,  which 
does  not  include  an  on-line  identifier,  produces  larger  path  deviations,  and  faster  rise-times. 


Larger  path  deviations  limits  the  LMFC  to  tasks  requiring  small  motions.  Potential 
performance  advantages  gained  from  a  coupled  controller  are  suggested  by  isolating  specific 
problems  associated  with  the  independent  adaptive  controllers. 


6.2.1.  Sensitivity  Analysis 
The  sensitivity  matrix,  J(X.),  is 


(61) 


and  the  transfer  function  matrix,  for  the  robot,  is 


(6.2) 


The  sensitivity  matricies  are  evaluated  to  address  the  following  questions: 

1.  Are  the  elements  in  (6.1)  continuous  and  non-degenerate  according  to  condition 
(2.6)? 

2.  If  the  controller  could  achieve  perfect  model  following  of  the  reference  features 
(i.e.,  eFp(k)  =  0  for  all  k),  what  would  be  the  predicted  Cartesian  path? 

3.  What  is  the  diagonal  dominance  organization  of  JWp,  which  determines  the 
feature/joint  coupling,  and  over  what  boundaries  does  this  organization 
remained  fixed? 

To  answer  these  questions,  numerical  maps  of  J  were  made  for  three  line  angles  (0Qbj  =  0,  45, 
75  degrees  )  and  over  the  range  of  relative  positions 

-4  <  Xrel  <  4  (in.)  and  0  <  Yrel  <  25  (in.)  in  1  in.  increments 

(The  range  of  angles  and  positions  correspond  to  those  used  in  the  simulation  examples 
evluated  in  Section  6.2.2  ).  A  typical  map  of  -Jfeat(X.rej)  is  shown  in  Figure  6-1.  The  matrix  is 
not  evaluated  at  points  where  the  line  is  hidden  or  is  gut  of  the  field  of-view  (OFV).  These 
points  are  indicated  by  a  on  the  maps.  Under  these  conditions,  the  matrix  elements  are 
continuous  and  non-degenerate. 


W  (s) 

p 


K 


s(rm.xS+1)2Wp 


K 


S(Tm-yS+1)2wP 


J(X)=-Jfea,(Xre,)  = 


ax 


cog 


ax 

ai 


(5.1) 


ax 


con 


rel 


av 

at 


(5.5) 


rel 


— snath  (5.2)  — eoath  (5.4) 

ax  ™ 


rel 


av, 


rel 


6.2.1 .1 .  Predicted  Path 


Evaluation  of  control  performance  includes  both  time  and  path  perfoimance.  Path 
performance  is  evaluated  relative  to  the  predicted  path,  or  trajectory,  that  would  result  if  the  I 

controller  could  achieve  perfect  model  following.  The  predicted  pathes  are  calculated  in  this 
section.  Two  approaches  are  available  for  deriving  these  pathes.  One  method  is  based  on 
numerical  evaluation  of  the  inverse  of  the  sensitivity  matrix  according  to: 
dX  =  J'1  df 

A  second  approach  is  to  analytically  solve  the  feature-to-Cartesian  space  mapping  (i.e.,  the 
interpretation  transformation).  Since  the  analytic  equations  for  the  Cartesian-to-feature 
space  mapping  (i.e.,  the  inverse  interpretation  transformation)  have  been  derived  (in  Section 
4.5),  the  later  approach  is  used  here.  The  feature  transformation  equations  (4.30)  and 
(4.31)  are  rewritten  as: 

X  fvp-(0.5l  sin#  i)2!^  +  [(0.51  sin#  .)2sin0  .cost?  .1  }  (6.3) 
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I  =1  (inch)  and  V=F  +V  . 
o  o  rel 

For  each  sampling  period,  (6.3)  and  (6.4)  are  solved  for  Xre|(k)  and  Yfe|(k),  as  a  function  of  the 
feature  reference  model  signals,  Xcog  ref(k)  and  leng(h  re((k).  The  cubic  equation  (6.4)  is  solved 
numerically  for  V,  and  the  resulting  Y  (  is  substituted  into  (6.3)  to  obtain  Xre|.  The  Cartesian 
path  is  then: 

X(k)  =  Xob,Xre,(k)and  Y(k)  =  Yobj  Y  -(k) 

The  relative  complexity  of  this  interpretation  process  for  even  this  2  DOF  system  illustrates  the 
potentiality  for  error  and  computational  complexity  which  would  be  required  in  a  position- 
based  system. 


The  visual  servoing  tasks  evaluated  in  this  section  are  represented  in  Figure  6-2.  For  each 


Figure  6-2:  2  DOF  Linear  Kinematic  and  Dynamic  Tasks 


156 


task,  the  predicted  path  (in  a  forward  trajectory)  is  displayed  in  Figure  6-3.  The  analysis 
demonstrates  that,  for  this  limited  degree-of-freedom  system,  the  predicted  paths  are  smooth 
and  exhibit  nearly  straight-line  trajectories.  Such  motion  is  satisfactory  for  many  applications. 
Had  this  2  DOF  system  yielded  paths  with  tortuous  trajectories,  then  it  would  not  be  very 
promising  to  expect  better  behavior  from  more  general  systems. 

6. 2. 1.2.  Feature/Joint  Assignment 

Selection  of  the  feature/joint  assignment  (Section  2.5.2. 1)  is  based  on  the  column 
arrangement  of  the  transfer  function  matrix  JW^  which  minimizes  D(JWp).  The  following 
derivation  illustrates  this  proceedure.  The  robot  transfer  function  matrix  in  (6.2)  is  written  as 

K,,(S)  0 

w  (s)  = 

L  o  k22(S) 


and  the  feature  sensitivity  in  (6.1)  is  written  as 
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The  overall  transfer  function  matrix  can  be  wriiten  as  either 
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The  feature  changes  <5f  =  [^^cog,^length]  ■  respect  to  the  control  signal  changes 
6u  =  [<5u , ,5u2]t,  become 


5U.h  L5u2 

J  CASE  1 


CASE  2 


°ln  Figure  6  3,  predicted  paths  for  Tasks  I  and  L  are  straight  lines,  and  the  paths  for  Tasks  E  and  F  have  path 
deviations  from  the  Y-axis  of  is  0.002  and  0  026  inches,  respectively. 
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For  Case  1,  the  feature/joint  assignment  would  be  Xcog«~X  axis  and  lenfi(h«— V-axis.  For  Case 
2,  the  assignment  would  be  Xcog<— Y-axis  and  length*— X-axis.  The  assignment  decision  is 
based  on  the  column  arrangement  which  minimizes  the  coupling;  i.e., 

if  D(J  Wp)<D(J  Wp)  then  select  assignment  in  Casel ;  (6.5) 

else  choose  Case  2 

Condition  (6.5)  leads  to 
Case  1 

ljni2^l  <  |i,2i2,i 
Case  2 


Maps  of  feature/joint  assignment  are  derived  from  sensitivity  maps.  An.  assignment  map, 

which  corresponds  to  the  sensitivity  map  of  Figure  6-1,  is  displayed  in  Figure  6-4.9  For 

example,  the  control  structure  for  the  assignment  at  Xfe|  =  {X|e),Vrol}T  =  {0,3}T  is  represented 

in  Figure  6-5.  Figure  6-4  indicates  that  the  feature/joint  assignment  may  require  switching 

during  a  control  task.  However,  upon  inspection  of  these  maps,  it  is  observed  that  the 

feature/joint  assignment  remains  fixed  over  large  regions  of  space,  and  there  are  no 

"islands"  where  the  assignment  changes,  but  there  is  a  distinct  switching  boundary.  Further, 

over  most  of  the  space,  Xcog  is  assigned  to  the  X-axis,  and  length  to  the  Y-axis.  For  0Qbj  equal 

to  zero,  the  assignment  remains  constant  over  all  space.  As  0obj  increases,  the  boundary 

shifts  to  the  left,  but  X  remains  assigned  to  the  X-axis  over  most  of  the  region.  These 
cog 

observations  are  in  contrast  to  those  for  the  nonlinear  kinematic  configuration  (in  Section 
6.3.2. 1)  for  which  the  assignments  do  not  remain  constant  over  large  regions  of  space. 


When  the  sensitivity  matrix  is  organized  in  a  diagonally  dominant  fashion,  the  coupling  (or 
off  diagonal)  elements,  can  change  by  a  larger  percentage  over  space  than  the  main  diagonal 
elements.  For  example,  if  the  camera  were  forced  to  move  along  a  straight  line  trajectory 
(represented  by  Task  H  in  Figure  6-2),  then  the  corresponding  percentage  change  in  each  of 
the  elements  is  plotied  in  Figure  6-6,  where 


%change= 


IJ(  -1  ,3)-J(  X,e,  ■  Yre,)l 


IJ(-1  .3)1 


X  100% 


It  is  noted  that  the  sensitivity  matrix  did  not.  in  general,  satisfy  the  classic  definition  of  dominance  in  (2.3).  For 

_ i ~  v  f  *  I  i : _ _  . i - t  i.nu 


example,  at  X,|e|  =  (1 ,3}  ,  classical  dominance  does  not  hold. 


Figure  6-5:  Feature/Joint  Assignment 


Figure  6-6:  Percentage  Change  In  Feature  Sensitivity 
For  other  tasks,  changes  in  sensitivity  can  be  even  larger  (e.g.,  in  Task  J,  J(2,2)  changes  by 
477%  or  by  a  factor  of  5,77).  The  significance  of  this  observation  is  that  it  is  expected  that 
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even  larger  sampling  to-bandwidth  ratios  will  be  required  for  identification  and  control  of 
these  coupled  image-based  systems.  When  using  independent  controllers,  the  coupling 
elements  can  be  viewed  as  external  disturbances.  Thus,  the  sampling  rate  must  be  high 
enough  to  reject  the  apparent  rapidly  changing  disturbances. 

In  most  of  the  simulations  presented  in  this  section,  the  feature/joint  assignment  is  fixed  to 

X  <— X-axis,  irrespective  of  diagonal  dominance  changes.  In  some  examples,  performance 
cog 

is  evaluated  when  the  feature/joint  assignment  is  switched  at  the  boundary  of  diagonal 
dominance  changes.  In  these  examples,  control  is  halted  at  the  boundary  and  the  feature 
assignment  is  switched.  When  the  assignment  is  switched,  the  identifier  parameters  must  be 
reset  or  initialized  to  the  values  used  for  that  particular  coupling.  Control  is  then  restarted 
with  reference  model  reinitialization.  In  these  examples,  D(k)  is  measured  during  control 
using  analytic  derivations  of  J(X  and  assuming  that  {X  Y  }  is  known. 

6. 2. 1.3.  Path  Constraint 

A  path  constraint  strategy,  for  this  configuration,  would  not  appear  to  be  required  since  the 
predicted  paths,  in  Figure  6-3,  approach  straight- lines.  Such  weil  behaved  paths  may  not 
result  in  more  general  and  higher  DOF  systems.  Even  for  this  simple  system,  perfect  model 
following  is  not  expected  because  of  transient  identification  errors  due  to  rapidly  changing 
gains,  measurement  delays,  and  use  of  independent  controllers.  Thus,  evaluation  of  a  path 
constraint  strategy  is  a  practical  exercise. 

A  path  constraint  approach,  based  on  the  hierarchical  control  strategy  presented  in  Chapter 
2,  is  evaluated.  The  hierarchy  is  represented  in  Figure  6-7,  where 
IX  'X  ( k)l 

%  A  X  =  ..Jon  desired - C0£L1_1  x  Q0% 

cog  Ax 

cog-max 

arid  the  percentage  feature  error  e  is  chosen  experimentally  to  be  e  =  10%  .  The  maximum 
feature  change,  AXcog  max,  >s  unity,  since  the  image  coordinates  are  normalized  to  the  image 
boundaries.  For  these  simulations,  Y-axis  motion  is  constrained  by  braking.  In  the  figure,  it  is 
assumed  that  the  Xcqij  feature  is  dominated  by  movement  along  the  X  axis  (i.e.,  all  areas 
marked  with  "o"  on  the  feature/joint  assignment  maps). 


(  £  =  10%) 


♦Control  only  X-axis 
W/  X  coy 

♦Restrict  Y-axis  by  brake 


♦Control  both  DOF 


Figure  6-7:  Hierarchy  for  2  DOF  Linear  Kinematic  System 
6.2.2.  Control  Evaluation 

The  visual  servoing  tasks  to  be  evaluated  are  represented  in  Figure  6-2.  Figure  6-2  also 
indicates  the  sensitivity  matrices  in  the  initial  and  desired  views,  and  the  associated  coupling 
index10.  Tasks  are  simulated  for  both  the  forward  (or  approach)  and  reverse  (or  backing- 
away)  directions.  The  tasks  are  selected  so  that,  in  the  desired  view,  the  camera  is  in  close 
proximity  to  the  object,  as  if  the  robot  were  going  to  acquire  the  object.  This  is  a  worse  case 
arrangement  since  J  changes  rapidly  as  the  camera  approaches  the  line. 

Each  task  is  simulated  at  two  sampling-to-bandwidth  ratios  by  fixing  the  bandwidth  to 
wn  =  9.5  sec'1,  and  setting  the  sampling  period  to  either  T  =  0.033  or  0.013  seconds11, 
corresponding  to 


°Fc  ;his  configuration  the  magnitude  of  W  ,  used  to  calculate  D(JW  ),  are  the  translational  stage  DC  gains 
K/2:rp  (Sections  4.3.3  and  4.4  4).  Since  the  gains  are  identical  for  both  stages,  and  Wp  is  diagonal.  D(JWp)  =  D(J) 

11  Due  to  open-loop  reference  model  time  constant  round  off,  the  closed-loop  bandwidth  is  u>n  =  9.4  sec  ,  when 
T  =  0.013 


fs/fBW  =  20,  when  T  =  0.033  sec. 
and 

fs/fBw  =  51,  when  T  =  0.013  sec. 

The  controller  paramters  are  summarized  Appendix  C.  After  each  initial  adaptive  learning  trial, 
a  fixed  LMFC  is  implemented  using  the  steady-state  estimated  plant  parameters  identified 
from  the  initial  adaptive  trial.  The  resultant  approach  trajectories,  at  the  larger  sampling-to- 
bandwidth  ratio  and  with  the  fixed  feature/joint  assignment,  are  shown  in  Figures  6-8,  6-9, 
and  6-10.  Each  plot  shows  the  adaptive  control  on  the  second  trial,  the  fixed  LMFC,  adaptive 
control  with  the  path  constraint  hierarchy,  and  the  predicted  path.  Each  plot  indicates  the 
rise-time  as  the  maximum  of  the  X-axis  and  Y-axis  position  rise-times.  Each  task  is  simulated 
for  3  seconds. 

The  following  observations  emerge  from  the  performance  evaluation  of  the  adaptive 
controller: 

1.  Stability  :  At  the  higher  sampling-to-bandwidth  ratio  (fs/fBW  =  51 )  and  for  the  the 
fixed  feature/joint  assignment,  all  simulations  are  stable,  wuh  acceptable 
transient  response,  and  zero  steady-state  feature  and  position  errors. 

At  the  lower  ratio  (f  /f(3W  =  20),  which  was  suitable  for  the  1  DOF  configuration, 
transient  response  degrades  when  the  tasks  are  run  in  the  reverse  directions,  and 
when  crossing  boundaries  of  diagonal  dominance  changes.  Problems 
associated  with  independent  adaptive  control  of  coupled  systems  are 
accentuated,  including  not  switching  the  feature/joint  assignment  at  boundaries 
of  diagonal  dominance  changes,  and  MRAC  sign  sensitivity.  The  MRAC  sign 
sensitivity  causes  momentary  changes  in  the  sign  of  the  controller  gain,  which 
causes  momentary  reversals  in  table  directions.  Examples  of  feature/joint 
reassignment  and  MRAC  sign  sensitivity  are  presented  in  Section  6.2.3. 

2.  Parameter  Estimation  :  When  the  line  stays  in  the  field-of-view,  steady-state 
identification  errors  go  to  zero.  The  transient  identification-errors  approach  zero 
before  steady-state  positions  are  reached  (i.e..  while  the  camera  is  still  in  motion), 
which  indicates  that  equivalent  SISO  models  of  the  coupled  plant  can  be 
identified.  Sufficiently  large  sampling-to-bandwidth  ratios  are  then  required  to 
track  dynamic  changes.  Transient  identification-errors  remained  for  all  examples. 

Type  0  plants  were  identified  in  the  steady-state,  and  some  of  the  estimated  zero’s 
are  non-minimum  phase.  This  demonstrates  the  need  for  a  control  penalty  and 
signal  biasing. 

At  the  higher  sampling-to-bandwidth  ratio,  adaptive  performance  in  the  second 
trial  is  not  significantly  improved  (relative  to  the  initial  trial)  due  to  the  higher 
sampling-to-bandwidth  ratio,  and  better  initializations  of  the  parameter  vector. 
Examples  of  tho  first  trial  performances  are  displayed  in  Figure  6-11. 
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Figure  6-8:  Linear  Kinematic  Trajectories  (f  /fBvy  =  51) 


In  the  second  trial  trajectories,  the  object  remains  in  a  stationary  position 
between  successive  trials.  If  the  object  is  allowed  to  assume  random  positions, 
should  the  identifier  be  reset  between  successive  trials?  To  examine  this 
question,  Tasks  J,  H  and  I  are  run  successively  without  resetting  the  identifier, 
and  with  the  identifier  initialized  to  the  values  learned  in  Task  E.  This  is  equivalent 
to  the  object  being  in  random  positions  between  successive  trials.  The  results  are 
displayed  in  Figure  6-12.  The  responses  are  not  as  smooth  as  if  the  controller 
had  been  reset  between  trials.  This  suggests  that,  for  a  margin  of  safety,  the 
controller  may  be  reset  between  successive  trials. 


X 


Figure  6-11:  Adaptive  Trajectories:  First  Trial 
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3.  Path  :  Smooth  motion  trajectories  are  observed  for  all  cases  evaluated  at  the 
higher  sampling-to  bandwidth  ratio  {fg/f BW  =  51 ) .  Such  motion  is  suitable  for 
tasks  requiring  small  corrective  motions.  Actual  paths  deviate  from  the  predicted 
trajectory,  and  the  maximum  path  deviations  are  summarized  in  Table  6- 1 . 


TASK 


Path  G 

E 

J 

H 

I 

L 

K 

F 

N 

M 

Deviation  Q  Q 

(  i  nclies ) 

0.006 

0.03 

0.04 

.  14 

^0.05 

0.  1 

0.23 

0.23 

0.83 

0.3 

(net)  \ 

(after  recovery  from 
s  i  «jn  sensitivity) 

T a ble  6- 1 :  Maximum  Path  Deviations 

The  tasks  requiring  relatively  large  distances  of  travel  (e.g..  Tasks  F,  M,  and  N) 
show  the  largest  deviations,  since  control  signal  saturation  and  larger 
identification  errors  occur  in  these  cases.  In  most  of  the  smaller  motion 
examples,  deviations  are  relatively  small.  Tasks  L  and  K  exhibit  relatively  larger 
deviations,  since  the  sensitivity,  J,  changes  by  a  relatively  large  amount,  which 
leads  to  larger  identification  errors.  In  addition,  the  initial  view  coupling  index 
(indicated  in  Figure  6-2)  for  Tasks  L  and  K  are  large  relative  to  values  for  the  other 
cases.  (The  result  observed  here  is  consistent  with  a  systematic  study  of 
performance,  as  a  function  of  the  coupling  index,  in  Chapter  7.)  In  Task  H,  the 
initial  path  oscillation  results  from  sign-sensitivity,  and  is  discussed  in  Section 
6.2.3. 1 

Actual  position  trajectories  are  not  always  monotonic.  If  smooth  motion 
trajectories  are  acceptable  for  path  performance,  then  monotonic  smooth  motion 
trajectories  are  desirable.  For  example,  when  independent  controllers  are  used, 
then  non  monotonic  motions  generally  result  when  initial  relative  displacement  is 
in  only  one  degree-of-freedom.  In  these  cases,  there  can  be  initial  feature  errors 
for  both  axes,  so  that  initial  motion  is  forced  in  both  axes  (e.g.,  Tasks  E,  I,  L,  and 
F).  Path  deviations  can  be  reduced  by  using  larger  sampling-to-bandwidth  ratios, 
but  typically  with  only  marginal  improvement.  For  example,  when  Task  I  is 
simulated  with  an  extremely  large  ratio  of  fs/fBW  =  21 3  (T  =  0.003  sec  and  «n  =  9.8 
sec  '),  the  maximum  deviation  is  0.03  inches,  compared  with  a  deviation  of  0.05 
inches  at  the  lower  sampling-to-bandwidth  ratio.  Thus,  SISO  control  of  feature 
coupled  systems  tend  to  have  path  errors  at  the  start  of  the  trajectory 
(independent  of  the  sampling-to-bandwidth  ratio),  which  accounts  for  the 
observed  transient  identification  errors.  An  MIMO  controller  would,  in  theory, 
produce  superior  path  performance. 

4.  Rise-Times:  Position  rise-times  are  sluggish  relative  to  the  predicted  response 
times.  For  example,  while  the  predicted  rise  time  is  typically  tf~  0.7  sec  for  the 
smaller  motion  tasks,  actual  times  ranged  from  0.04  lo  1.27  seconds  (except  for 
Task  L).  Saturation,  control  penalties,  and  transient  identification-errors  (which 
appear  to  over  estimate  the  gains)  produce  transient  response  errors.  To 
illustrate  the  error  due  to  control  penalty,  Tasks  E  and  J  are  simulated  without 
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measurement  delay  (i.e.,  nd  =  0)  so  that  the  controller  pole  bound  can  be 
increased  to  y  =  0.99  .  The  rise-times  deceased  to  0.81  seconds,  and  the 
remaining  time-domain  error  arises  from  transient  identification  error. 

With  measurement  delay  included,  a  larger  control  penalty  (ie.,  smaller  pole 
bound)  is  required.  While  no  attempt  is  made  to  optimize  y  for  the  examples 
included  in  the  dissertation  (i.e.,  y  -  .65  for  all  examples  and  for  all  f_/fBVV),  it  is 
observed,  that  as  fs/fBW  increases,  y  can  also  be  increased.  For  example,  if  Task 
J  is  simulated  with  y  =  0.75.  and  with  nd  =  1,  the  rise  time  decreases  to  0.82 
seconds,  without  degradation  of  path. 

It  appears  that  Task  L  exhibits  exceptionally  poor  time  response  (i  e.,  tr  =  2. 1 1  sec 
for  adaptive  control,  compared  with  a  predicted  rise-tme  tf  =  .73  sec)  as  a  result  of 
the  relatively  large  changes  in  3len  lh/dY  and  dX.co^/dX  and  large  value  of 
coupling.  For  example,  in  Task  L,  the  initial  view  coupling  index  is  D  =  .95,  and 
J(2,2)  changes  by  a  factor  of  8.9.  In  Task  E,  for  which  the  adaptive  and  predicted 
rise  times  are  .9  seconds  and  .68  seconds,  respectively,  the  initial  view  coupling 
is  D  =  .0008,  and  J(2,2)  changes  by  a  factor  of  2.2. 

Finally,  the  time  response,  when  moving  toward  the  line,  is  faster  than  when 
moving  in  the  opposite  direction.  This  observation  is  consistent  with  the 
predicted  path  rise-times. 

5.  Path  Constraint:  During  hierarchy  level  transitions,  the  MRAC  controller  gains,  for 
each  MRAC.  are  fixed  (i.e.,  run  as  LMFC)  to  their  current  values  (or  6  sampling 
periods  following  the  transition.  It  is  observed  that  this  approach  could  improve 
the  transient  response  during  the  transitional  interval.  It  appears  that  by 
permitting  the  additional  DOF  (i.e.,  the  Y-axis)  to  accelerate,  before  identification 
proceeds,  feature  changes,  and  thus  gain  estimates,  are  not  dominated  by  the 
axis  already  in  motion.  The  number  of  fixed  cycles  is  determined  experimentally. 

The  hierarchical  strategy  minimizes  trajectory  deviations  when  initial  relative 
displacement  is  predominantly  in  the  X  direction.  At  the  opposite  extreme,  there  is 
no  effect  in  cases  like  Task  E  where  initial  displacement  is  only  in  the  Y  direction. 
In  intermediate  cases,  such  as  in  Tasks  M,N,J,  and  H,  the  hierarchy  degrades  the 
path  relative  to  the  predicted  path,  but  at  least  drives  it  in  a  controlled  monotonic 
fashion  toward  the  X-axis.  In  Task  G,  where  the  object  is  not  centered  in  the 
image,  the  hierarchy  accentuates  the  path  deviation  required  to  drive  the  feature 
center  of-gravity  error  to  zero. 

In  general,  arbitrary  use  of  the  this  hierarchy  may  or  may  not  minimize  path  non¬ 
monotonic  trajectory  deviations.  However,  if  the  object  is  centered  in  the  image, 
then  a  smooth  path,  biased  toward  the  X-axis,  is  expected. 


Figures  6-8,  6-9,  and  6-10  also  display  the  fixed  LMFC  response.  (The  fixed  controller  is  only 
run  at  the  higher  sampling-to-bandwidth  ratio  f  /f  =51.)  The  following  observations 
emerge: 


1.  Stability:  With  the  exception  of  Task  M,  the  fixed  controllers  are  stable,  with  good 
transient  responses,  and  zero  steady-state  feature  and  position  errors.  In  Task  M, 
the  line  goes  OFV,  and  control  stops.  In  this  case,  the  LMFC  is  not  adequate  even 
with  the  relatively  large  bandwidth.  This  suggests  that  a  fixed  control  strategy  may 
not  be  suitable  for  tasks  requiring  large  motions  where  the  sensitivity,  J,  changes 
by  large  amounts. 

2.  Path:  Each  task  exhibits  smooth  path  motion.  With  the  exception  of  Task  FI,  the 
LMFC  produces  deviations  from  the  predicted  path  which  are  larger  than  the 
MRAC  path  deviations.  Task  Ft,  which  is  discussed  in  Section  6.2.3. 1 ,  exhibits  an 
overall  LMFC  response  that  is  superior  to  the  MRAC,  since  the  fixed  controller  is 
not  sign-sensitive. 

An  additional  experiment  is  conducted  in  which  Tasks  L,  J,  H  and  I  are 
implemented  with  the  same  fixed  controller.  The  fixed  controller  gains  are  learned 
in  Task  E,  and  the  results  are  displayed  in  Figure  6-13. 


X 

Fixed  From  TASK  C 

Figure  6-13:  Single  Fixed  Controller  for  Multiple  Tasks 


In  these  examples,  path  deviations  are  larger  than  those  resulting  when 
individually  learned  LMFCs  were  used  (e.g.,  in  Figure  6-13,  the  deviation  for  Task 
L  is  0.35  inches,  compared  with  0.15  inches  for  the  same  task  in  Figure  6  10  and 
in).  This  suggests  that  a  single  fixed  controller  cannot  achieve  uniform  response 
over  a  broad  range  of  tasks. 

3.  Rise- Times:  Generalizations  cannot  made  concerning  LMFC  vs.  MRAC  rise- 
times.  The  LMFC  rise-times  are  faster  than  the  MRAC  in  seven  out  of  ten  trials. 
The  adaptive  controller  tends  to  be  slower  than  the  fixed  controller  due  to 
transient  identification  errors. 


6.2.3.  Independent  Adaptive  Control  Problems 

In  the  path  performance  summary  in  Section  6.2.2,  it  was  explained  why  transient 
identification  errors  will  generally  occur  for  SISO  control  of  coupled  systems.  In  this  section, 
other  problems  associated  with  independent  adaptive  control  of  a  feature  coupled  system  are 
clarified.  These  problems,  which  can  affect  transient  response  performance,  include: 

•  Adaptive  Controller  sign-sensitivity;  and 

•  Feature/joint  assignment. 

A  coupled  controller  would  not  exhibit  similar  problems.  Algorithms  for  coupled  controllers, 
and  their  relative  computational  complexities,  are  advanced  in  Chapter  0  for  future  research 
evaluation. 

6. 2. 3.1.  Sign-Sensitivity 

It  is  demonstrated,  in  Section  5.3,  that  the  MRAC  controller  is  sign-sensitive.  That  is,  the 
identifier  sets  the  sign  of  the  estimated  gains  to  ensure  a  stable  negative  feedback  system 
(assuming  control  of  a  SISO  process).  In  a  coupled  system,  the  SISO  identifier  sets  the  gain 
signs,  of  estimated  equivalent  SISO  models,  to  drive  each  identificaticu-error  to  zero.  It  is 
shown  below  that,  momentary  changes  in  sign  can  degrade  performance  by  causing  an  axis 
to  reverse  direction  and/or  decelerate.  When  the  estimated  sign  switches  again,  it  takes  time 
for  the  system  to  recover  from  the  axis  reversal  due  to  its  momentum  in  the  opposite  direction. 
At  the  lower  sampling-to-bandwidth  ratio,  it  takes  a  longer  time  to  recover  since  the  axis  can 
travel  further  and  gain  more  momentum  during  the  longer  sampling  interval.  At  the  higher 
ratios,  the  system  recovers  faster  from  subsequent  identification-errors. 

Sign-sensitivity  effects  system  performance  in  Task  H.  In  this  example,  the  Y-axis  starts  out 
in  the  correct  direction,  but  then  momentarily  reverses  its  motion.  The  trajectory  and  the 

A 

estimated  gain  for  the  length  feature,  b,  ((,nglh(k).  are  displayed  in  Figures  6-14  and  6-15  for 
the  two  sampling  to-bandwidth  ratios.  At  the  lower  sampling-to-bandwidth  ratio  (T  =  0.033  sec 
and  fg/fBW  =  20),  the  effects  of  sign-sensitivity  are  accentuated.  During  the  initial  phase  of 
control,  both  axis  start  out  in  the  correct  direction  (i.e.,  in  such  a  away  as  to  reduce  the 
feature  errors).  Over  the  initial  sampling  interval,  the  axes  move  by 

AX  ~  -0.025  inches  and  AY~  0.01  inches 

The  change  in  the  length  feature,  as  specified  by  the  sensitivity  map  in  Figure  6- 1  becomes: 
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Figure  6- 14:  Sign-Sensitivity  (T  =  0.033  and  fg/fBW  =  20) 
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Figure  6- 1  5:  Sign  Sensitivity  (T  =  0.013  and  fs/fBW  =  51) 
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Since  AY>0  and  Alerig[h<0,  at  that  instant  ,  b,  is  forced  to  be  negative,  driving  the  Y-axis  in 
the  opposite  direction. 

Strategies  which  can  be  used  witli  the  independent  MRAC  controller  to  counter  these 
effects,  include: 

A 

•  Clamp  sign:  experiments  are  conducted  where  the  sign  of  the  gain  term  b,  is 
forced  to  that  of  the  corresponding  sign  of  the  diagonal  term  of  the  sensitivity 
matrix.  This  leads  to  smooth  and  stable  performance,  but  the  generality  of  this 
strategy  needs  further  investigation. 

•  Higher  sampling-to-bandwidth  ratios:  the  higher  the  ratio,  the  faster  the  recovery; 
and 

•  Fixed  control:  sign  insensitive. 

6. 2. 3. 2.  Feature/Joint  Reassignment 

Trajectories  which  cross  boundaries  of  change  in  diagonal  dominance  correspond  to  Tasks 
K  and  L.  In  these  cases,  a  fixed  feature  joint  assignment  (i.e. ,  X  axis*—  Xcog,  Y-axis*—  lenglh) 
produce  stable  systems  with  acceptable  transient  responses,  at  the  higher  sampling-to- 
bandwidth  ratio  (T  =  0.013  sec  and  fs/fBW  =  51).  At  the  lower  sampling-to  bandwidth  ratio 
(T  =  0.033  sec  and  fs/fbw  =  51).  poor  transient  response  can  cause  the  line  to  go  OVF.  The 
effects  of  switching  the  feature/joint  assignment  at  the  boundaries  of  predicted  reassignment 
are  examined  in  this  section.  The  experiments  demonstrate  that  potential  improvements  in 
performance  can  be  gained  by  assigning  features  to  minimize  D(k),  and  thereby  reduce 
coupling.  If  a  trajectory  crosses  a  boundary  of  change  in  diagonal-dominance,  without 
reassignment  of  features,  path  performance  can  degrade  and  the  system  can  be  unstable, 
especially  at  lower  sampling-to-bandwidth  ratios.  The  effects  are  more  pronounced  for 
systems  with  nonlinear  kineamtics  (in  Section  6.3),  independent  of  the  sampling-to-bandwidth 
ratio. 

The  feature/joim  ^assignment  simulations  are  implemented  using  the  analytic  derivations 

of  J(Are|).  and  assuming  that  X.rc|  is  known.  A  fixed  feature/joint  assignment  corresponding  to 

X-axis*— I  Y-axis*— X  is  also  evaluated, 

ength  cog 

The  response  for  Task  K,  at  the  lower  sampling-to-bandwidth  ratio  (T  =  0  033  sec  and 
fs/fBW  =  20)  and  with  the  fixed  feature/joint  assignment,  (corresponding  to  X-axis<—  XCOQ, 
Y-axis*—  I  tf)),  is  shown  in  Figure  6-16.  The  system  is  unstable  with  the  fixed  the  feature/joint 
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Figure  6-16:  Feature/Joint  Reassignment  (T  =  0.033  and  f  /fBW  =  20) 
assignment.  The  line  goes  OFV  in  the  initial  trial,  and  in  second  trial  (not  shown)  exhibits  even 
poorer  response  (i.e.,  oscillation)  before  the  line  goes  OVF.  Figure  6-16  also  includes  the 
response  for  the  second  trial  of  an  experiment  for  which  feature/joint  reassignment  is 
implemented  at  the  switching  boundary.  In  this  experiment,  control  is  halted  when  the  camera 
reaches  the  boundary  (and  it  is  assumed  that  control  can  be  stopped  instantaneously  by 
manual  braking).  At  this  point,  the  current  state  of  the  identifier,  including  the  estimated 
parameter  vector  and  gain  matrix,  are  held  for  subsequent  use.  The  feature/joint  assignment 
is  then  switched,  and  the  identifier  is  reinitialized,  since  the  dynamics  associated  with  each 
feature  from  the  first  part  also  switch.  Control  then  proceeds  until  the  completion  of  the  task. 
At  the  end  of  the  task,  the  identifier  state  is  again  held  for  use  on  subsequent  trials  in  this 
region.  The  task  is  run  again,  but  with  the  identifier  initialized  to  the  previously  held  states. 
With  this  strategy,  the  line  stays  in  the  field -of- view,  the  system  is  stable,  and  the  trajectory 
exhibits  acceptable  transient  performance.  Gy  switching  the  feature/joint  assignment,  the 
coupling  in  the  desired  view  decreases  from  D  =  0.95  to  D  -  0.39  .  The  improved  stability  is  a 
result  of  the  feature/joint  reassignment,  and  not  just  the  stop  and  then  start  up  again  action. 
This  is  verified  by  stopping  the  system  at  the  boundary  of  change  of  diagonal  dominance  and 
then  restarting  it,  without  switching  the  feature/joint  assignment.  In  this  example  the  line  still 
goes  OFV.  Improved  performance  is  thus  achieved  by  the  apparent  'eduction  in  coupling 
gained  by  the  feature/joint  reassignment. 
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The  feature/joint  reassignment  experiment  was  also  implemented  at  the  higher  sampling-to- 
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Figure  6-17:  Feature/ Joint  Reassignment  (T  =  0.0 1 3 sec  and  fs/fBW  =  51) 
bandwidth  ratio  (see  Figure  6-17).  There  is  improvement  in  path  performance,  but  the  rise¬ 
time  is  larger  due  to  model  reinitialization  at  the  switching  point. 

Task  K  is  also  evaluated  with  the  opposite  fixed  feature/joint  assignment  corresponding  to 
X-axis«— fength,  Y-axis-* Xcug.  This  assortment  forces  D=i.62  in  the  initial  view,  which  is 
almost  two  orders  of  magnitude  larger,  than  for  the  opposite  assignment.  This  system  is 
unstable,  even  at  the  higher  sampling-to-bandwidth  ratio  (fs/fQW).  It  could  be  stabilized  by 
running  it  at  a  very  high  sampling-to-bandwidth  ratio,  by  reducing  the  sampling  period  to 
T  =  0.003  and  fa/fRW  =  220,  but  the  transient  response  is  poor  (i.e.,  oscillation). 

Task  L  is  also  evaluated  using  the  feature/joint  reassignment  strategy.  While  the  path 
deviation  is  slightly  larger,  the  rise- time  decreases.  A  fixed  assignment  corresponding  to 
X  axis*— I,  Y-axis*— Xcog  is  also  implemented.  The  system  is  stable,  but  the  transient  response 
is  poor  (i.e.,  oscillation). 


While  the  switching  strategy  used  in  these  experiments  is  static,  in  the  sense  that  control  is 
halted  at  the  switching  boundary,  the  potential  advantage  of  reducing  coupling  oy 
feature/join'  assignment  is  demonstrated.  A  method  for  on-line  estimation  of  J  or  JW()  is 
proposed,  in  CnupterO,  for  feature/joint  reassignment  strategies. 


This  concludes  the  evaluation  of  the  2  DOF  linear  kinematic  and  dynamic  configuration.  The 


second  part  of  this  chapter  extends  this  evaluation  to  the  nonlinear  kinematic  and  dynamic 
robot. 

6.3.  Nonlinear  Kinematics  and  Dynamics 

The  purpose  of  this  section  is  to  evaluate  IBVS  control  for  the  nonlinear  kinematic  and 
dynamic  system  of  Configuration  2  (Figure  4-2).  The  articulated  robot  arm  places  additional 
demands  on  the  controller,  including  nonlinear  and  coupled  kinematics  and  dynamics.  For 
the  control  algorithm  to  be  suitable  for  an  image-based  system,  the  controller  muot  be 
adequate  as  a  joint-level  controller.  The  first  part  of  this  section  evaluates  the  enhanced  SP 
MRAC  algorithm  at  the  joint-level,  and  demonstrates  its  ability  to  acvhieve  stable  systems  with 
good  transient  response. 

The  image-based  control  system  yields  performances  which  are  comparable  to  the  linear 
kinematic  system  when  tasks  are  limited  to  small  motions.  The  performance  differs  from  the 
linear  kinematic  system  in  two  key  respec 

•  For  tasks  requiring  large  motions,  independent  control,  of  the  kinematically 
coupled  system,  can  produce  relatively  large  path  deviations;  and 

•  For  the  linear  kinematic  configuration,  a.  fixed  feature/joint  assignment  is 
adequate.  In  contrast,  the  nonlinear  kinematic  configuration  requires  a 
feature/joint  reassignment  strategy  for  general  implementations,  except  when  the 
required  task  motion  is  small. 

6.3.1 .  Control  of  Nonlinear  Dynamics 

Investigations  by  other  researchers  (Section  2.3. 1.3)  have  demonstrated  the  ability  of 
independent  adaptive  controllers  to  perform  at  the  joint  level.  These  studies  have  not  payed 
attention  requirements  for  physical  implementation,  including  computational  complexity, 
controller  saturation,  noise,  and  measurement  and  computational  delays.  Morris  and 
Neuman  [Morris  81]  developed  the  SISO  hyperstable  SP  MRAC  taking  account  of  these 
requirements.  They  evaluate  this  algorithm  for  one  DOF  linear  systems,  with  slowly  varying 
parameters  (without  measurmerit  delay),  and  suggest  that  this  controller  can  be  extended  to 
control  a  nonlinear  and  coupled  robot  [Morris  81].  Before  proceeding  with  the  evaluation 
image-based  control,  this  algorithm  is  illustrated  as  a  joint  level  controller  (with  measurement 
delay  included).  Exhaustive  testing  and  evaluation  of  joint-level  control,  which  is  not  included 
in  this  dissertation,  remains  a  subject  for  future  research. 
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Figu  re  6- 1 8:  Adaptive  Control  of  a  2  DOF  Robot  Arm 

Joint  level  control  of  the  2  DOF  robot  arm  is  represented  in  Figure  6-18.  With  the 
measurement  delay,  the  independent  controller  algorithms  are  implemented  according  to  the 
outline  in  Appendix  C,  with  the  exception  that  the  scale  factors  are  given  by: 

SFT  =  0  033=,0° 


SFt  .  oq3  -  1 000 

and,  the  identifier  is  initialized  according  to  Table  6-2. 


P(0)  P  1  hresh  ^  U  > 


Diag  [10  10  10  10]  1000  [10  10  1.1  0.1]T 


0.003  IDiag  [.1  .1  .01  .01]  10  [1  1  1.5  -0.5]' 


Table  6-2:  Identifier  Initialization:  Joint  Level  Control 
Some  of  the  experiments  are  implemented  without  the  measurement  delay.  For  n 
controller  is  implemented  according  to: 

•  Control  Signal:  (3.10) 
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•  Identifier:  (3.4), (3.5),  and  (3.6) 

•  Control  Penalty:  (A. 2 1 )  with  y  =  0.99 

•  SP  Reference  Model:  Appendix  A.1 

There  are  simulation  errors  because  of  the  numerical  integrations  which  comprise  the 
computer  simulation  of  the  nonlinear  dynamics  The  relative  magnitudes  of  error  are 
evaluated  by  simulating  the  step-response  of  each  joint,  with  the  other  joint  locked  in  a 
stationary  position.  In  this  experiment,  each  joint  acts  as  a  constant  linear  system,  and  the 
estimated  parameter  vector  should  equal  the  actual  vector  for  the  linear  systems  derived  in 
Section  4.4.5. 1.  This  experiment  is  run  at  two  sampling  periods,  T  =  0.033  sec  and  T  =  0.003 
sec.  The  results  are  shown  in  Table  6-3,  where  the  estimated  vector  is  given  as  its  steady-state 
value  after  two  consecutive  learning  trials.  The  percentage  error  is: 

A 

%error=  (^§s)  X  100% 

P 

While  the  errors  are  relatively  small,  the  experiments  in  Chapter  5,  for  which  the  dynamics  are 
simulated  by  analytic  equations,  produce  zero  error. 

6.3.1 .1 .  Joint-Level  Control  Examples 

While  exhaustive  testing  and  evaluation  of  joint-level  control  is  not  performed,  initial 
experimentation,  using  random  arm  configurations,  produce  stable  systems  with  good 
transient  response.  An  example  is  presented  which  demonstrates  the  ability  of  ihe  MRAC  to 
achieve  satisfactory  performance  under  demanding  conditions  which  include: 

1 .  High  joint  velocities  and  accelerations; 

2.  Control  signal  saturation;  and 

3.  Measurement  delay. 

The  experiment  is  set-up  as  follows: 

•  Sampling  Period:  The  sample  peried,  T  =  0.003  sec,  corresponds  to  the  value 
used  in  present  state-of-art  robots. 

•  Reference  Model  Bandwidth:  A  large  bandwidth,  wn  =  44.5  sec'1,  produces  high 
joint  speeds  (for  step-inputs),  and  produces  control  signal  saturation.  This 
bandwidth  corresponds  to  a  sampling  to-bandwidth  ratio  of  fs/fLJW  =  47. 

•  Task:  Run  four  consecutive  adaptive  trials  (i.e. ,  without  reinitialization  of  the 
identifier  between  trials),  with  step-inputs  and  varying  pay  loads  according  to: 
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This  particular  set  of  sampling  periods,  bandwidth,  and  arm  configurations  are  also  useful 
since  they  correspond  to  values  which  are  evaluated,  in  Section  6.3.3,  under  imaged-based 
control. 

The  position  responses  are  displayed  in  Figure  6-19,  and  the  control  signals  and  torque 
components  for  the  second  and  third  trials12  are  displayed  in  Figures  6-20  and 
6-21  respectively.  The  components  of  torque  are  defined  in  Section  4.4.2  as: 

Joint  1 

D))^).  Inertial 

D)0<?2 :  Inertial  Coupling 

D  112^1  ^2  '  Coriolis 
D122022 :  Centrifugal 

Joint  2 

D  :  Inertial  Coupling 
oJ2 :  Inertial 

D2nt?i2:  Centrifugal 

The  torque  plots  verify  that  the  dynamics  include  non-negligible  coupling  and  nonlinear 
components.  Examples  for  two  trials  are  shown  to  demonstrate  the  different  contributions 
which  arise  from  arm  configuration  dependence.  In  the  second  trial,  the  inertial  coupling 
terms  contribute  significantly  to  the  net  torque  during  the  initial  accelerations.  As  Q  2 
increases,  D12  (Section  4.4.2)  decreases  with  the  the  cosine  of  $2-  Thus,  during  the  final 
deceleration,  there  is  a  smaller  contribution  of  inertial  coupling.  The  opposite  effects  are  then 
observed  in  the  third  trial.  The  Coriolis  and  centrigual  torques  are  non  negligible  in  both 
cases. 

,2Each  is  run  tor  3  seconds,  but  only  0.5  or  1  second  of  data  are  shown  for  clarity  and  detail. 


In  these  examples,  the  full-parallel  errors  exhibit  large  transient  tracking  errors.  The 
maximum  errors  are  summarized  in  Table  6  4  for  these  and  similar  experiments  which  are 
discussed  below.  In  Case  f,  the  large  error  is  due  primarily  to  saturation  and  control  penalty, 
since  identification  errors  are  relatively  small.  The  experiment  is  run  again  (Case  2),  but  at  a 
smaller  bandwidth  (wn  =  9.81  sec'1)  to  reduce  the  total  time  of  saturation.  The  error  is  reduced 
by  half,  and  most  of  the  remaining  error  results  from  the  large  control  penalty  required  by  the 
measurement  delay.  It  is  observed  that  larger  pole  bounds,  and  thus  smaller  penalties,  could 
be  used  for  these  joint  level  controllers.  The  effects  of  the  control  penalty  are  demonstrated 
by  running  the  experiment  without  any  measurement  delay  (i.e.,  nd  =  0),  and  with  the  pole 
bound  set  to  y  =  0.99  .  In  this  example  (Case  3),  there  is  a  dramatic  decrease  in  the  full- 
parallel  error.  Momentary  saturation  remains,  and  small  identification  errors  remain.  To 
reduce  further  controller  saturation,  two  experiments  are  run  with  smaller  motions.  At  the 
larger  bandwidth  (Case  4),  saturation  remains,  and  at  the  smaller  bandwidth,  there  is  no 
saturation.  The  identification  errors  are  thus  responsible  for  the  remaining  tracking  error. 
The  small  errors  suggest  that,  for  high  precision  tracking,  further  compensation  (e.g., 
feedforward  control)  may  still  be  required. 


A  fixed  l.MFC  is  also  simulated  for  the  four  consecuuvo  tasks  in  (G.6).  Each  task  is  run  with 
the  identifier  fixed  to  the  estimated  parameter  vector  learned  in  Trial  3.  The  LMFC 
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Table  6-4:  Jt  ■  it  Level  Control:  Maximum  FP  Errors 
performance  (not  displayed)  is  negligibly  different  from  the  MRAC.  Performances  may  be 
comparable  since  saturation  occurs  for  a  significant  percentage  of  time  for  both  the  MRAC 
and  LMFC.  Since  the  control  signal  is  the  same  for  each  controller  during  this  period,  the 
position  response  must  also  be  the  same  during  the  saturation  interval.  To  evaluate  the 
contripution  of  controller  saturation,  a  smaller  motion  task  is  simulated  (with  0 40-  *45  dog 
and  02. 95-*90  deg)  to  reduce  saturation  effects.  The  identifier  is  fixed  to  values  learned  trom 
the  first  adaptive  trial  in  this  configuration.  In  this  case  there  is  only  momentary  saturation, 
but  both  fixed  and  adaptive  performances  are  still  comparable.  Another  possibility  for 
comparable  performance  is  that,  at  the  high  bandwidth,  the  sensitivity  of  the  fixed  controller 
to  apparent  external  disturbances  and  parameter  variations  is  small.  To  evaluate  this,  the 
small  motion  task  is  run  again,  at  the  smaller  bandwidth  w  =  9.81  sec  \  The  identifier  is  fixed 
to  values  learned  from  the  first  adaptive  trial  in  this  configuration,  and  at  the  smaller 
bandwidth.  The  position  responses,  displayed  in  Figure  6-22,  show  that  the  MRAC  performs 
better  at  the  lower  bandwidth,  consistent  with  the  results  of  Chapter  5.  The  LMFC  shll  exhibits 
good  transient  response  and  the  overall  performance  is  not  significantly  poorer  than  that  of 
the  MRAC.  Another  factor  accounting  for  to  the  satisfactory  LMFC  response  may  be 
attributed  to  the  tieavy  gearing,  which  attenuates  the  changes  in  torque  reflected  back  to  the 
motor,  although  it  does  not  change  their  relative  contributions. 

While  further  research  is  required  for  conclusive  evaluations  of  adaptive  and  fixed  joint  level 
control,  those  limited  experiments  have  the  following  significance: 
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6.3.2.  Sensitivity  Analysis 


The  sensitivity  matrix,  in  (2.7),  for  the  nonlinear  kinematic  configuration  is 

J<4)  =  -J, J„„<4> 

and  the  feature  sensitivity  matrix,  J(eat,  is  defined  in  (6.1).  The  arm  jacobian,  Jarm  in  (2.5),  is 


From  (4.13),  the  direct  kinematic  solution,  K(£)  in  (2.5)  is 
x  =  r2cos(0  ,  +  02)  +  r^os^j 
y  =  r2sin(/y  t  ^2)  +  ^sintf, 

and  includes  the  tool  or  camera  mounting  transform. 

Maps  of  the  sensitivity  matrix  are  made  for  the  visual  servoing  tasks  which  are  presented  in 
this  section.  The  tasks,  represented  in  Figure  6-23,  are  equivalent  to  linear  kinematic 
examples  with  respect  to  both  the  initial  and  desired  relative  Cartesian  coordinates.  The 
feature  sensitivity  map  for  the  smaller  motion  tasks  (i.e.,  all  except  Tasks  V  and  W)  is 
displayed  in  Figure  6  24.  For  this  set  of  tasks,  the  matrix  elements  are  continuous  and 
nondegenerate.  When  the  arm  is  in  other  configurations,  which  are  kinematically  degenerate 
(i.e,  det[Jarm]  =  0  ),  the  system  is  degenerate.  For  example,  when  the  arm  is  fully  extended 
(i.e.,  #2  =  0  ),  clet[Jarm]  =  0,  which  indicates  the  transition  from  the  "elbow-ups"  to  "eibows- 
down"  joint  configurations.  At  the  joint  level,  these  represent  non-unique  solutions,  but  not 
affect  the  final  Cartesian  task  coordinates.  With  respect  to  effects  on  dynamic  control,  such  a 
system  cannot  (in  general)  be  uncoupled,  even  when  using  a  coupled  controller.  Control 
evaluation  of  tasks  which  might  traverse  such  boundaries  are  not  presented  in  the 
dissertation 

For  the  smaller  motion  tasks,  sensitivity  changes  are  similar  in  magnitude  to  the  equivalent 
linear  kinematic  examples.  Over  larger  distances  (e  g..  Tasks  W  and  V),  the  changes  are 
significantly  greater  in  the  nonlinear  kinematic  configuration,  as  illustrated  in  Table  6-5,  which 
compares  these  changes  for  Task  W  and  the  equivalent  linear  kinematic  Task  N.  The  table 
indicates  the  change  in  sensitivity  from  the  initial  to  the  desired  view,  and  compares  it  both  as 
a  percentage  and  ratio.  By  either  measure,  the  nonlinear  kinematic  configuration  exhibits 
significantly  larger  changes.  It  is  therefore  expected  that  even  higher  sampling  to-bandwidth 
ratios  will  be  required  for  control  of  large  motion  nonlinear  kinematic  tasks. 
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Table  6-5:  Nonlinear  vs.  Linear  Kinematic  Sensitivity  Change 
6.3.2. 1.  Feature/Joint  Assignment 

The  nonlinear  kinematic  configuration  is  dynamically  nonlinear,  and  a  transfer  function 
matrix  Wp  can  be  only  be  obtained  by  linearizing  the  dynamics  about  a  nominal  trajectory,  or 
by  estimating  equivalent  linear  input-output  transfer  functions.  For  the  purpose  of  calculating 
a  coupling  index,  a  linear  model  is  generated  by  neglecting  the  Coriolis  and  centrifugal 
torques,  and  assuming  the  inertias  are  constant  for  any  paiticular  arm  configuration.  For 
small  motion  tasks,  the  Coriolis  and  centrifugal  torques  are  shown  (in  Section  6.3.3.1)  to  be 
negligible.  For  large  motion  tasks,  neglecting  these  terms  is  not  an  issue  since  the  SISO 
controllers  are  shown  to  be  not  acceptable  (in  Section  6.3.3.2)  even  for  control  of  linear 
dynamic  models.  The  transfer  function  matrix,  from  (4.14),  (4.15),  (4.16),  and  (4.17),  is: 
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To  calculate  the  coupling  index  D(JWp),  the  magnitude  of  JWp  is  derived  using  the  DC  gains 
Kn,  K)2,  K21>  and  K22.  This  is  justified  since  the  maximum  time  constant  of  the  characteristic 
equation  (6.7)  for  the  arm  configurations  in  Figure  6-23  is  51  msec,  while  the  observed 
position  rise-times  range  from  0.18  to  0.28  seconds.  (While  the  feature  rise  times  are  not 
recorded,  feature  rise-times  are  even  larger,  according  to  the  analysis  in  Section  5.4.) 


The  feature/joint  assignment  map  is  also  displayed  in  Figure  6-24.  For  these  examples,  the 
specified  assignments  are: 


cog 


8. 


and 
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over  the  entire  displayed  region,  and  this  assignment  also  holds  for  all  line  angles  (<?obj).  In 
contrast  to  the  linear  kinematic  configuration,  this  assignment  does  not  hold  over  large 
regions.  For  other  arm  configurations,  the  opposite  assignment  is  observed.  For  example,  in 
the  sensitivity  and  assignment  maps  displayed  in  Figure  6-25,  there  is  a  boundary  where  the 
assignment  becomes 


cog 


- 8 „ 


and 


Wigth*-  ^1 

As  the  line  angle  decreases,  the  later  assignment  holds  over  the  entire  displayed  space.  In 
contrast  to  the  linear  kinematic  configuration,  a  fixed  feature/joint  assignment  is  not 
predicted  for  large  regions  of  control.  The  performance  when  crossing  such  a  boundary,  and 
its  effect  on  implementation  using  independent  adaptive  controllers,  is  discussed  in  Section 
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6. 3. 2. 2.  Path  Constraint 


Implementation  of  a  path  constraint  strategy  for  the  nonlinear  kinematic  configuration 
requires  sensitivity  analysis  of  the  arm  jacobian  according  to  Section  2.5.3  .  This  analysis 
shows  which  joint  level  degrees-of-freedom  dominate  motion  in  Cartesian  space. 

The  path  constraint  strategy  is  analyzed  for  the  tasks  represented  in  Figure  6-23.  A  map  of 
the  arm  jacobian  and  of  the  Cartesian/joint  assignment  is  displayed  in  Figure  6-26  for  the 
closer  range  motions.  This  assignment  map  is  derived  by  organizing  Jarm  to  minimize  D  (i.e., 
in  (2.9),  JWp  is  replaced  with  Jarni).  For  this  particular  arm  configuration,  0 1  dominates  X,  and 
from  the  feature  sensitivity  analysis  of  Jfeat.  X  dominates  Xcog.  The  hierarchy  for  this 
configuration  is  shown  in  Figure  6-27. 

6.3.3.  IBVS  Evaluation 

The  visual  servoing  tasks  evaluated  are  represented  in  Figure  6-23.  The  smaller  motion 
tasks  (i.e.,  all  except  Tasks  V  and  W)  are  discussed  (in  Section  6.3.3.1),  seperately  from  the 
larger  motion  examples  (in  Section  6.3.3.2),  since  significantly  different  results  are  observed 
in  each.  Examples  of  tasks  which  have  trajectories  that  cross  boundaries  of  diagonal 
dominance  are  discussed  (in  Section  6.3.3.). 

Preliminary  simulations,  which  are  implemented  with  a  reference  model  bandwidth  of 
«n  =  9.5  sec'1  and  T  =  0.033  sec  (or  f3/fBW  =  20).  are  stable,  but  exhibit  poor  transient 
response.  In  contrast,  when  the  equivalent  linear  kinematic  tasks  are  implemented  with  these 
values,  smooth  trajectories  result  (in  the  forward  trajectory  and  across  regions  where  fixed 
feature  joint  assignment  is  predicted).  The  problem  is  isolated  to  insufficient  sampling-to- 
bandwidth  ratio  required  for  suitable  identification,  and  it  appears  that  the  added  kinematic 
coupling  is  responsible.  To  demonstrate  this,  the  following  experiments  are  simulated  for 
Task  S: 

1.  To  verify  that  the  nonlinear  dynamics  are  not  the  source  of  the  problem,  direct 
joint-level  control  of  the  robot  arm  (in  this  configuration  and  with  the  same 
bandwidth  and  sampling  period)  exhibits  accepatble  transient  response. 

2.  To  assure  that  the  combination  of  nonlinear  dynamics  and  kinematics  do  not 
cause  the  problem,  the  visual  servoing  task  is  simulated  with  uncoupled  and 
linear  dynamics  according  to  Section  4.4.5. 1  .  The  response  exhibits  poor 
transient  response. 


3.  A  fixed  LMFC,  which  does  not  include  an  on-line  identifier,  is  implemented  using 
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Figure  6-27:  Hierarchy  for  2  DOF  Nonlinear  Kinematic  Configuration 

the  steady-state  parameter  vector  learned  from  an  initial  adaptive  trial.  The  results 
are  shown  in  Figure  6-28.  While  the  adaptive  system  displays  a  poor  transinet 
response,  in  both  the  first  and  second  trials  (second  trial  shown),  the  LMFC 
produces  a  dramatically  improved  response  (i.e.,  reduced  oscillation,  and  close 
to  monotonic  motion). 

4.  It  is  verified  that  the  sampling  period,  T  =  0.033  sec,  is  not  too  small  relative  to  the 
robot  dynamics  (i.e.,  the  time  constants  associated  with  the  characteristic 
equation  (6.7)  are  18.1  msec  and  48.9  msec,  at  02  =  9O.2  deg),  and  that  the 
sampling-to-bandwidth  ratio  is  the  important  parameter.  The  task  is  run  with  the 
nonlinear  dynamic  simulation,  and  with  a  smaller  bandwidth  un  =  4.39  sec'1,  so 
that  fs/fBW  =  43.  This  results  in  an  acceptable  transient  response. 

The  significance  of  these  experiments  is  that  a  sampling-to-bandwidth  ratio  of  at  least 
fs/fBW—  45  is  required  for  adaptive  control  with  nonlinear  kinematics.  In  addition,  these 
experiments  suggest  that  a  fixed  controller  is  more  suitable  for  control  when  physical 
implementation  requires  smaller  f3/fQW  ratios. 
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Figure  6-28:  Fixed  vs.  Adaptive:  T  =  0.033  sec  and  f/fQW  =  20 
6.3.3. 1 .  Small  Motion  Trajectories 

This  section  includes  a  summary  of  the  control  performance  for  the  small  motion 
trajectories  represented  in  Figure  6-23.  The  evaluation  is  based  on  simulations  using  T  =  0.003 
sec  and  «n  =  44.51  sec'1  (fs/fQW  =  47).  The  smaller  sampling  period  permits  a  larger  banwidth 
to  be  specified,  which  leads  to  faster  speeds  and  accelerations.  This  accentuates  the 
magnitudes  of  the  coupled  and  nonlinear  dynamics,  testing  the  MRAC’s  ability  to  operate 
under  these  demanding  conditions.  The  fixed  feature/joint  assignment  for  this  configuration 
is  described  in  the  sensitivity  analysis  in  Section  6.3.2. 1 .  The  experiments  conducted  for  each 
task  match  those  for  the  linear  kinematic  configuration.  That  is,  after  each  initial  adaptive 
learning  trial,  a  fixed  LMFC  is  implemented  using  the  steady-state  estimated  parameter  vector. 
The  path  constraint  strategy  is  implemented  with  the  controller  fixed  for  6  sampling  periods 
during  the  level  transitions  (Section  6.2.2.).  The  resultant  trajectories  are  displayed  in  Figures 
6-29  and  6-30.The  torque  components  for  the  adaptive  control  of  Task  P  are  displayed  in 
Figure  6-31. 13  This  representative  example  demonstrates  that  there  is  significant  inertial 
coupling,  but  the  Coriolis  and  centrifugal  torques  are  negligible  for  these  shorter  excursion 
tasks. 


While  each  trial  is  run  for  3  seconds,  only  0.4  seconds  are  shown. 
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Figure  6-29:  Nonlinear  Kinematic  Trajectories  (T  =  .003; 
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Figure  6-31 :  Torque  Components  for  TASK  Q  (MRAC) 


Observed  performance  of  the  small  motion  trajectories  qualitatively  match  those  of  the 
linear  kinematic  configuration.  In  summary,  these  observations  are: 

1 .  Stability:  All  tasks  simulations  are  stable,  with  accepatble  transient  response,  and 
zero  steady-state  feature  and  positional  errors. 

2.  Parameter  Identification:  Steady-state  identification  errors  go  to  zero.  Transient 
errors  approach  zero  before  the  steady-state  positions  are  reached. 

3.  Path:  In  each  case,  smooth  trajectories  are  achieved,  which  deviate  from  the 
predicted  path.  Path  deviations  are  not  consistently  smaller  or  larger  than  for  the 
equivalent  linear  kinematic  configuration. 

Tasks  Z  and  R  exhibit  relatively  large  deviations,  which  result  from  the  nonlinear 
kinematics  (and  not  tho  nonlinear  dynamics  or  the  combination  of  nonlinear 
dynamics  and  kinematics).  To  verify  this,  Task  Z  is  simulated  with  a  linear  and 
uncoupled  dynamic  plant  model  according  to  Section  4.4.5. 1  (with  rm  =  0  058  sec 
and  rm  =  0.027  sec)  with  similar  results. 

4.  Rise-times:  The  adaptive  rise-times  are  sluggish,  relative  to  the  predicted 


199 


response.  As  noted  for  the  linear  kinematic  trials,  no  attempt  was  made  to 
optimize  the  controller  pole-bound,  y,  which  is  kept  constant  throughout  the 
experimentation.  In  practice  the  control  penalty  can  be  reduced  as  the  sampling- 
to-bandwidth  ratio  increases,  resulting  in  faster  rise-times. 

5.  Path -Constraint:  When  the  object  is  centered  in  the  image,  the  path  constraint 
strategy  produce  smooth  controlled  motion  toward  the  X-axis.  While  this 
technique  is  useful  for  this  particular  set  of  arm  configurations,  more  exhaustive 
testing  is  required  to  determine  its  application  in  other  configurations  and 
geometries.  Limitations  of  this  strategy  are  accentuated  in  the  larger  motion 
studies. 


For  the  fixed  LMFC  control: 

1.  Stability:  With  the  exception  of  Task  Q,  the  LMFC  systems  are  stable,  with  good 
transient  response,  and  zero  steady-state  errors.  In  Task  O,  there  is  a  small 
oscillation  toward  the  end  of  the  trajectory. 

2.  Path:  Maximum  path  deviations  exceed  the  adaptive  path  deviations. 

3.  Rise-times:  No  generalizations  can  be  made  concerning  fixed  vs.  adaptive  rise 
times.  Consistent  with  the  linear  kinematic  examples,  the  LMFC  tends  to  be  faster 
that  the  MRAC  in  most  of  the  trials. 

6. 3. 3. 2.  Large  Motion  Travel 

The  experiments  presented  in  this  section  demonstrate  that  IBVS  control  of  nonlinear 
kinematic  configurations,  for  tasks  requiring  large  motions,  may  not  be  suitable  unless 
coupled  controllers  and  extremely  large  sampling-to- bandwidth  ratios  are  used..  The 
evaluations  also  show  that  fixed  control  and  the  path  constraint  hierarchy  strategies  are  not 
suitable  for  such  tasks.  With  the  exception  of  the  fixed  controller  applicability,  these 
conclusions  oppose  those  for  the  linear  kinematic  configuration.  What  are  the  possible 
differences  between  these  configurations  which  account  for  these  observations?  With 
respect  to  the  linear  kinematic  configuration,  the  nonlinear  kinematic  configuration: 

1 .  Exhibits  significantly  larger  changes  in  sensitivity  over  the  longer  distances 
(Table  6-5); 

2.  Has  nonlinear  and  coupled  dynamics;  and 

3.  Is  kinematically  coupled. 

Of  these  differences,  it  is  shown  below  that  the  coupled  kinematics  ultimately  limit  the  use  of 
independent  controllers  in  an  IBVS  system. 

Tasks  V  and  W  were  simulated  with  values  of  f*/fQW  and  T  used  for  the  small  motion  studies. 


TT-r-r-rT'-iTT'T^T? 
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The  simulation  of  Task  V  produces  a  smooth  stable  motion,  with  a  deviation  from  the 
predicted  path  of  1.21  inches  on  the  first  trial  (and  1.01  inches  on  the  second  trial).  However, 
in  Task  W,  the  line  keeps  going  OFV  due  to  poor  stability.  For  both  examples,  the  predicted 
feature/joint  assignment  is  verified  to  be  fixed  at  Xcog«— 0t.  using  analytic  measurements 
along  the  trajectory.  The  ability  to  control  Task  V  may  be  attributed  to 

1.  Its  change  in  J2  >  which  is  two  orders  of  magnitude  smaller  than  in  Task  W;  and 

2.  Its  smaller  coupling  in  the  initial  view  (i.e.,  DTask  v  =  -0.002  vs.  DTisk  w  =  0.657  . 

(Chapter  8  includes  a  systematic  study  of  performance  as  a  function  of  coupling.) 

In  neither  task  would  a  fixed  LMFC  keep  the  line  in  the  field  of  view. 

Further  evaluation  of  Task  W  isolates  kinematic  coupling  as  a  source  for  independent 
control.  It  is  verified  (in  Section  6.3.1. 1)  that  the  nonlinear  dynamics,  for  this  arm 
configuration,  can  at  least  be  controlled  at  the  joint  level.  Next,  it  is  verified  that  the 
combination  of  nonlinear  dynamics  and  kinematics  is  not  a  problem.  Task  W  is  simulated  with 
linear  and  uncoupled  dynamic  plant  models  (with  rm  )  *  rm  2  =  0.1  sec  in  Section  4.4.5. 1.),  and 
the  line  still  goes  OFV.  The  sampling-to-bandwidth  ratio  fs/f0w  is  increased  to  415  by  keeping 
T  =  0.003  sec  and  decreasing  the  bandwidth  to  wn  =  5.03  sec'1.  The  resulting  trajectory,  on  the 
second  trial,  is  displayed  in  Figure  6-32. 14  The  trajectory  exhibits  a  relatively  large  deviation 
from  the  predicted  path.  The  deviation  is  due  partly  to  the  presence  of  control  signal 
saturation  in  the  first  joint,  but  not  the  second.  Thus,  8y  which  dominates  motion  in  the  X 
direction,  cannot  drive  the  camera  toward  that  axis.  However,  it  is  verified  that  the 
contribution  of  saturation  to  this  deviation  is  minimal,  by  simulating  the  task  at  a  lower 
bandwidths.  At  the  lower  bandwidths,  saturation  does  not  occur,  but  the  camera’s  path  still 
exhibits  a  relatively  large  deviation  from  the  X-axis  of  5.3  inches  (compared  with  5.8  inches  at 
the  higher  bandwidth). 

The  hierarchical  approach  could  not  be  used  to  offset  these  large  path  deviations.  Failure  of 
the  hierarchical  strategy  in  the  nonlinear  kinematic  configuration  results  from  the  inherently 
poor  trajectory  performance.  Over  the  larger  distance  of  travel,  the  constrained  joint  (02) 
tended  to  be  cyclically  activated,  and  then  braked,  to  keep  AXcofl  within  the  error  bounds. 
Frequent  switching,  back  and  forth,  between  levels  of  the  hierarchy  results  in  jerky  motion. 
Such  motion  cannot  be  tolerated  in  a  practical  robot  because  of  the  vibrations  and 


14 

The  first  trial  goes  OFV  since  the  parameter  vector  initialization  is  too  large,  and  P(0)  is  increased  to 
Diag(1  1  1  1],  The  net  time  indicated  for  this  trajectory  (3  seconds)  coitespond;.  to  the  maximum  time  that  the 
simulator  can  run  at  T  =  0.003  sec. 
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Figure  6-32:  Nonlinear  Kinematics:  Large  Motion  Travel  (f«/fBW  -  415) 
mechanical  resonances  which  would  result.  (In  the  smaller  motion  tasks,  there  is  only  a  single 
transition  between  levels.)  Further,  in  these  large  motion  examples,  it  is  difficult  to  select  a 
combination  of  controller  parameters  (e.g.,  P(0),  e,  and  the  number  of  fixed  transition  cycles) 
which  would  even  produce  a  stable  response  under  these  frequent  switching  conditions. 

These  experiments  demonstrate  that,  independent  IBVS  control  of  nonlinear  kinematic 
configurations,  which  requirie  large  motions,  can  produce  large  path  deviations.  While  path 
deviations  may  be  tolerable  for  for  tasks  requiring  small  motions,  over  larger  regions  of 
control  there  are  increased  risks  of  collisions  with  other  obstacles  in  the  region,  and  it  is  more 
difficult  to  keep  the  object  within  the  camera's  field-of-view.  Implementations  of  IBVS  control 
of  nonlinear  kinematic  robots  will  require  coupled  controllers,  using  relatively  large  sampling- 
to-bandwidth  ratios.  Alternatively,  independent  controllers  may  be  used  if  the  robot 
kinematics  are  first  uncoupled  by  using  a  dynamic  image  based  "Look  &  Move”  structure. 

6. 3. 3. 3.  Feature/Joint  Reassignment 

Analysis  and  evaluation  of  the  2  DOF  linear  kinematic  configuration  suggest  that  a  fixed 
feature/joint  assignment  is  satisfactory  for  control  of  such  a  system.  In  contrast,  the 
sensitivity  analysis  in  Section  6.3.2,  and  the  example  presented  in  this  section  demonstrates 
that,  when  individual  degrees  of  freedom  do  not  coincide  with  the  camera's  frame  (e.g.  a 
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nonlinear  kinematic  configuration),  a  fixed  feature/joint  assignment  is  not  generally  suitable. 
The  sensitivity  analysis  shows  that  the  arm  jacobian  dominance,  and  therefore  overall 
sensitivity  dominance,  changes  with  the  arm  configuration.  This  section  evaluates  motion 
across  a  boundary  of  diagonal  dominance  change.  The  experiments  suggest  that,  when 
crossing  such  boundaries,  a  fixed  feature/joint  assignment  can  only  be  used  if  the  required 
motion  is  relatively  small. 

A  task  which  requires  crossing  a  boundary  of  change  in  diagonal  dominance  is  represented 
in  Figure  6-33.  The  task  is  simulated  with  three  strategies:15 

1.  Fixed  feature/joint  assignment :  02«— XcQg  (corresponds  to  assignment  predicted 
in  desired  view); 

2.  Feature/joint  reassignment:  at  boundary  of  diagonal  dominance  change;  and 

3.  Fixed  feature/joint  assignment :  0t«-X  (corresponds  to  assignment  predicted 
in  initial  view). 


Figure  6-33:  Nonlinear  Kinematic  Feature/ Joint  Reassignment  Task 
The  resulting  trajectories  for  the  first  two  strategies  are  displayed  in  Figure  6-34.  The 
feature/joint  reassignment  trajectory  exhibits  a  smaller  path  deviation.  In  the  third  strategy 
(not  displayed),  which  is  satisfactory  for  the  configurations  evaluated  in  Section  6.3.3. 1,  the 

15These  simulations  use  the  controller  parameters  specified  for  the  small  motion  tasks  in  Section  6.3.3. 1. 
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Figure  6-34:  Feature/Joint  Reassignment  (Nonlinear  kinematics) 
line  goes  OFV  due  to  large  path  deviations.  For  all  cases,  the  fixed  LMFC  causes  the  line  to 
go  OFV  due  to  large  path  deviations. 

It  would  not  have  been  suitable  to  implement  this  task  using  a  feature/joint  assignment 
based  on  the  sensitivity  in  the  initial  view,  since  the  line  goes  OVF.  In  practice  without  an 
on-line  estimation  of  JWpl  the  sensitivity  in  the  initial  view  is  all  that  can  be  determined  from 
measurements  preceeding  the  start  of  control.  This  issue  is  discussed  in  Chapter  7.  If  the 
required  motion  is  even  smaller  (e.g.,  small  corrective  tasks),  measurement  of  JWp 
preceeding  the  start  of  control  may  be  satisfactory.  For  example,  the  task  in  Figure  6-33  is 
redefined  so  that  the  initial  and  final  arm  posjtions  are  separated  by  only  0.3  inches  (i.e.,  02 
47— +46  degrees,  and  0 1  is  43  degrees  in  both  initial  and  final  views  respectively.)  When  this 
task  is  simulated  with  0t*~X  ,  the  resulting  trajectory,  which  still  crosses  the  diagonal 

dominance  boundary,  is  stable  and  displays  an  approximately  straight  line  motion. 


6.4.  Summary 

The  purpose  of  this  two  part  chapter  is  to  evaluate  independent  adaptive  control  of  coupled 
two  DOF  systems.  Visual  servoing,  of  .a  line  in  space  problem,  uses  two  basic  configurations: 

1. A  linear  and  uncoupled  kinematic  and  dynamic  robot,  for  which  the  robot's 
degrees-of- freedom  coincide  with  the  camera’s  reference  frame;  and 

2.  A  nonlinear  and  coupled  kinematic  and  dynamic  robot. 
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While  image-based  control  is  emphasized,  preliminary  evaluation  of  joint-level  control  of  the 
nonlinear  and  coupled  plant  dynamics  is  also  included.  To  the  best  of  the  author’s  knowledge, 
this  is  the  first  demonstration  of  an  enhanced  MRAC  (i.e. ,  SP  MRAC,  with  control  penalty  and 
measurement  delay)  for  robot  control. 

The  main  conclusion  which  emerges  from  this  evaluation  is  that  an  image-based  "teach-by- 
showing"  strategy,  using  an  adaptive  control  approach,  can  be  used  to  control  robots.  Since 
this  approach  cannot  be  used  to  control  path,  it  appears  to  be  best  suited  for  precision  tasks 
which  require  small  corrective  motions  (e  g.,  precision  assembly,  for  parts  aquistion  and 
mating  alignment).  However,  predicted  postion  trajectories,  for  at  least  this  simple  line  in 
space  problem,  are  smooth  and  approach  straight  line.  This  motivates  continuing  future 
research  to  evaluate  more  general  configurations. 


While  independent  control  of  the  coupled  systems  achieve  stable  performance,  with 
acceptable  transient  response,  and  zero  steady-state  error,  potential  limitations  of  this 
approach  include: 

•  Independent  controllers  produce  deviations  from  the  predicted  path.  While 
steady-state  errors  go  to  zero,  deviations  arise  from  transient  identification-errors 
occuring  during  the  intial  phases  of  control.  Path  deviations  are  relatively  small, 
with  the  exception  of  the  large  motion  nonlinear  kinematic  tasks.  For  these  tasks, 
indepenent  control  is  judged  not  to  be  suitable  due  to  increased  risks  of  obstacle 
collison,  and  the  requirement  of  extremely  large  sampling-to-bandwith  ratios 
(which  may  not  be  suitable  for  a  practical  implementations.) 

Independent  joint-level  control  of  a  dynamically  nonlinear  and  coupled  robot, 
using  an  enhanced  SP  MRAC  and  with  measurement  delay,  is  stable  with 
acceptable  transient  response.  The  control  problem  associated  with  the  large 
motion  nonlinear  kinematic  tasks  is  then  isolated  to  the  nonlinear  kinematics  (and 
not  to  the  dynamics). 

•  Independent  control  requires  a  feature/joint  reassignment  strategy  for  a  general 
system.  While  a  fixed  assignment  is  adequate  for  the  linear  kinemati  configuration 
(using  a  sufficiently  large  f</fBVV).  the  articulated  arm  configuration  requires  a 
reassignment  strategy,  unless  motion  is  constrained  to  small  movements. 
Boundaries  of  feature/joint  reassignment  are  successfully  predicted  by 
minimizing  the  coupling  index  D  (2.9). 

•  Since  the  MRAC  is  sign-sensitive,  independent  control  of  a  coupled  system  can 
cause  momentary  large  reversals  in  the  axis  direction.  Large  sampling-to- 
bandwidth  ratios  can  minimize  these  effects, 

These  limitations  suggest  potential  performance  advantanges  which  can  be  gained  by  using 
coupled  controllers. 
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Evaluation  of  a  path  constraint  hierarchy  shows  that  arbitary  use  of  this  hierarchy  may  or 
may  not  minimize  nonmonotonic  path  deviations.  This  strategy  produces  a  smooth  controlled 
path  biased  toward  the  X-axis,  except  for  the  large  motion,  nonlinear  kinematic  tasks.  For  this 
configuration,  the  large  path  deviations  could  not  be  offset  because  it  is  difficult  to  select  a 
combination  of  controller  parameters  which  would  yield  a  stable  system  under  the  frequent 
switching  conditions,  and  the  motion  is  too  jerky  for  practical  systems. 

Finally,  the  MRAC  is  used  to  derive  fixed  LMFC  controllers.  Relative  to  the  adaptive 
approach,  the  fixed  controllers  exhibit  better  transient  response  at  lower  sampling-to- 
bandwidth  ratios,  and  tend  to  have  faster  rise-times.  They  produce  larger  path  devaitions;  are 
only  suitable  for  small  motion  the  tasks  which  have  small  sensitivity  changes;  and,  a  single 
LMFC  is  not  suitable  for  a  broad  range  of  tasks. 

Only  two  control  features  were  available  for  the  2  DOF  configuration.  In  Chapter  7,  IBVS 
control  is  extended  to  a  three  DOF  system,  and  control  is  evaluated  for  multiple  candidate 
features.  Evaluation  of  a  single  task,  with  different  control  features,  provides  a  consistent  and 
systematic  study  of  performance  as  a  function  of  the  coupling  index  D(JWp). 
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Chapter  7 

Evaluation:  Three  DOF 

7.1 .  Overview 

The  purpose  of  this  chapter  is  to  evaluate  adaptive  image-based  SISO  control  of  the  3  DOF 
configuration  described  in  Section  4.2.3.  The  simulations  described  extend  adaptive  image- 
based  control  to  a  multi-DOF  system  which  models  a  real  configuration.  The  evaluation 
focuses  on  the  feature  selection  process  described  in  Section  2.5.2.  The  diverse  path  and 
time  performances,  resulting  from  using  different  feature  subsets,  emphasize  the  importance 
of  feature  selection.  The  coupling  index  D(JWp)  is  demonstrated  to  be  a  suitable  discriminant 
function  for  selecting  among  candidate  feature  subsets.  By  selecting  features  which  minimize 
this  index,  improved  system  performance  (including  both  path  and  time  response)  is  achieved 
as  D(JWp)  decreases.  These  results  are  consistent  with  those  observed  in  Chapter  6  for  the 
related  issue  of  feature/joint  assignment,  which  uses  D(JWp)  to  minimize  apparent  system 
coupling.  The  suitabilty  of  using  this  index  as  a  measure  of  coupling  becomes  an  important 
aspect  for  implementations  of  general  image-based  systems,  which  will  have  to  select  and 
assign  features. 

The  evaluation  also  illustrates  the  hierarchical  and  fixed  control  strategies.  The  hierarchical 
approach  not  only  constrains  path  motion,  but  also  facilitates  the  use  of  SiSO  adaptive 
control  by  making  performance  less  sensitive  to  coupling.  Adaptive  learning  trials  can  be 
used  to  synthesize  SISO  fixed  controllers,  whose  performance  begins  to  degrade  as  changes 
in  sensitivity  become  larger. 

This  chapter  is  organized  as  follows.  In  Section  7.2,  the  candidate  features  are  defined,  and 
measurement  of  the  system  sensitivies  is  described.  The  small-signal  evaluation,  in  Section 
7.3,  uses  tasks  requiring  small  motions  so  that  changes  in  the  magnitudes  of  the  feature 
sensitivity,  J,  and  thus  changes  in  the  coupling  index,  D(JWp),  are  also  small  over  the 
trajectory.  This  provides  a  convenient  task  for  analyzing  and  evaluating  adaptive  control 


performance  as  a  function  of  coupling  since  D(JWp)  remains  essentially  constant.  Larger 
motion  tasks,  with  significant  chages  in  J,  are  evaluated  in  Section  7.4  to  investigate  the 
suitabilty  of  selecting  features  based  on  coupling  in  the  initial  view,  and  also  to  show 
examples  of  fixed  and  hierarchical  control.  The  controller  parameters,  for  the  simulations 
described  in  this  chapter,  are  summarized  in  Appendix  D. 

7.2.  Candidate  Control  Features 

The  feature  selection  process  (in  Section  2.5.2)  asks:  How  should  a  subset  of  n  control 
features  be  selected  (n  =  3),  from  a  set  of  m  features,  where  m>n?  In  Section  2.5.2,  it  was 
proposed  that  the  selection  may  be  based  on  minimization  of  D(JWp).  In  this  section,  m 
candidate  features  (where  m  =  7)  are  defined  for  the  control  evaluation  in  Sections  7.3  and 
7.4.  Measurement  of  the  corresponding  sensitivities  is  then  discussed. 

The  cube  (in  Figure  4-13)  is  sitting  in  a  stable  position  on  top  of  the  turntable  (i.e.,  one  of  the 
cube’s  sides  is  fiat),  and  its  image  is  displayed  in  Figure  7-1. 


Figure  7-1:  Cube  Image 

In  the  figure,  each  plane  or  node16,  is  referenced  by  the  displayed  number  (e.g.  node  6,  4,  or 
1).  It  is  assumed  that  the  vision  system  can  distinguish  the  image  regions  associated  with 
each  node  (i.e.,  connected  regions  [Agin  79]).  The  vision  system  can  extract  the  image  area, 
A.,  and  centroid,  (\og.,ycog.)>  of  each  i,h  node.  Combinations  of  these  features,  such  as 
relative  areas: 

16 

The  origin  of  node  is  telatcd  to  a  graph  structure  representation  of  the  image  [Sanderson  82) 
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can  also  be  considered  for  selection.  It  is  initially  assumed  that  the  features  from  only  nodes  4 
and  1  are  available,  and  the  candidate  features  are: 


rel-1,2’ 


rel-2,1'  1+2'  cog-1' 


and  X 


cog-2 


There  are  thus  p(7,3)  =  210  candidate  subsets  of  ordered  features.  To  limit  the  number  of 
combinations  which  are  evaluated  in  the  dissertation,  feature  subsets  are  selected  which 
include  one  of  each  type  feature:  a  center-of-gravity,  an  area  or  sum  of  areas,  and  a  relative 
area  feature.  This  combination  is  chosen  because  the  2  DOF  evaluation  (in  Chapter  6)  has 
already  demonstrated  that  the  COG  and  length  of  a  line,  analogous  to  the  centroid  and  area  of 
a  plane,  can  be  used  to  control  depth  and  position.  The  relative  area  feature  is  added  in 
consideration  of  the  additional  orientation  DOF.  Feature/joint  assignment  is  described  (in 
Section  7.2.2)  after  the  discussion  of  sensitivity  measurements  (in  Section  7.2.1). 


7.2.1 .  Sensitivity  Measurements 


Preceding  the  start  of  control,  it  is  straightforward  to  measure  the  magnitude  of  JWp  (i.e., 
IJ1AM)  in  the  initial  perceived  image.  With  this  measurement,  the  coupling  index,  D(JWp),  can 
be  calculated  for  selection  and  assignment  of  an  initial  feature  subset.  The  magnitude  of  JWp 
can  be  measured  by  independently  evaluating  J  and  IWpl.  The  robot  transfer  function  matrix 
is 


Wp(s)  = 
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s(rm_xs+:)2irp 


S(rm  yS+  1)  2t rp 
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360 
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(7.1) 


The  magnitude  of  Wp  is  evaluated  by  assuming  that  the  the  open-loop  plant  time  constants, 
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r  ,  are  small  relative  to  the  closed-loop  time  constants.  This-is  justified  since  the  closed  loop 
position  rise-times,  observed  in  the  simulation  examples,  range  from  0.16  to  1.4  seconds'7, 
while  the  plant  time  constants  range  from  0.041  to  0.091  seconds.  When,  IWpl  is 
approximated  by  its  low  frequency  gain  characteristics: 
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— ' M - )  0  0 
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(7.2) 


This  assumption  simplifies  the  calculation  of  IWpl  when  the  exact  dynamic  model  is  not 
known.  In  this  case,  each  term  in  (7.2)  can  easily  be  be  measured  as  the  magnitude  of  the 
impulse  response  for  each  joint.  For  the  experiments  presented  in  this  chapter,  the  values 
used  for  Km,  p,  and  tj,  were  derived  in  Chapter  4.  In  (7.2),  since  u  1  is  a  common  factor,  the 
evaluated  radian  frequency  is  set  to  co  =  1.  For  higher  closed-loop  bandwidth  systems,  it 
would  be  appropriate  to  evaluate  (7.1)  at  the  closed  loop  bandwidth  «n. 


The  sensitivity  matrix  is 
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and  is  determined  by  sequentially  moving  each  DOF  by  a  small  displacement  and  then 
measuring  the  accompanying  feature  changes: 
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The  feature  control  variable  rise-times,  while  not  recored,  are  even  larger  according  to  the  analysis  in  Section 
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The  magnitude  of  the  joint  displacements  (Ax, Ay, A 0)  are  selected  by  succesive  iterative 
reduction  until  successive  values  of  each  element  of  J  satisfies 

~ — —  X  100%  <  0.01  % 

Jj 

In  (7.3),  the  dimensions  of  each  element  are  functions  of  only  the  joint  variable,  since  the 
features  are  normalized  to  the  screen  and  are  dimensionless.  Each  element  of  IJWpl  thus  has 
the  same  units  (volt  1).  This  is  imperative  to  be  able  to  compare  the  relative  magnitudes  of  the 
sensitivities. 

Since  the  magnitude  of  the  sensitivities  can  change  from  the  initial  to  the  final  image,  so  can 
D(JWp),  and  the  selected  feature  subset  can  change  along  the  trajectory.  Ideally,  it  would  be 
convenient  if  the  relative  relationships  between  the  coupling  index  of  each  subset  remained 
constant  along  the  trajectory.  Then,  only  off-line  measurements  of  the  sensitivities,  made 
preceding  the  start  of  control,  would  be  sufficient.  The  2  DOF  evaluation  has  already 
demonstrated  that  feature/joint  assignment  can  change  during  control  for  a  nonlinear 
kinematic  system.  The  3  DOF  evaluation  shows  that  the  selected  features  may  also  change. 
Thus,  in  a  completely  general  image-based  system,  IJWpl  might  be  estimated  on-line.  To 
evaluate  the  use  of  D(JWp)  to  discriminate  among  several  feature  candidates,  small  motion 
tasks  will  be  specified  so  that  J  and  D(JWp)  do  not  change  significantly  over  the  trajectory. 
Thus,  measurement  of  the  intial  view  couplings  are  suitable.  The  use  of  intial  view  coupling 
measurements  for  larger  motion  tasks  are  evaluated  in  Section  7.4  . 


7.2.2.  Feature/Joint  Assignment 

All  simulations  are  implemented  using  fixed  feature/joint  assignments  corresponding  to 


X  <— X-axis 
cog 

A  <-Y-axis 
A  _.<-0-axis 


For  each  candidate  feature  subset  evaluated,  it  is  verified  that  these  are  the  assignments 
predicted  according  to  minimization  of  D(JWp).  A  case  where  these  assignments  do  not  hold 
will  also  be  discussed. 


The  evaluation  in  Chapter  6  demonstrated  that  minimization  of  D  can  provide  a  suitable 
feature/joint  assignment  for  a  2  DOF  system.  For  three  and  higher  DOF  systems,  the  question 
arises:  If  JWp  is  organized  to  minimize  D(JWp),  but  partitioned  (2x2)  matrices  of  JWp  are  not 
each  minimized  independently,  can  control  problems  arise?  For  the  particular  3  DOF 
simulations,  evaluated  in  this  dissertation,  such  organizations  do  not  occur.  That  is,  it  was 
verified  that  the  three  partitioned  matrices,  which  result  by  canceling  one  DOF  and  the 
corresponding  feature  on  the  diagonal  of  the  IJWpl  matrix,  each  exhibit  a  minimum  in 
coupling.  The  partitioned  matrices  are: 
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(7.7) 


where 

K  360K 

K  =K  = — ^  (in/volt)  and  K..  = - 23  (deg/volt) 

x  y  2-rrp  "  20tj 

Future  research  must  verify  whether  minimization  inconsistencies  between  the  entire 
sensitivity  matrix  and  partitioned  (2x2)  sections  will  occur  for  other  configurations.  And,  if  so, 
will  these  limit  the  use  of  independent  controllers? 
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7.3.  Small  Signal  Evaluation 

In  this  section,  a  task  requiring  small  motions  is  specified  so  that  changes  in  the  magnitudes 
of  the  sensitivity,  J,  and  thus  coupling  index,  D(JWp),  are  also  small  over  the  trajectory.  The 
task  is  specified  to  have  a  net  displacement  of  AX  =  AY  =  0.1  inches  for  the  translational 
stages,  and  A0  =  1  degree  for  the  rotational  stage.  In  these  examples,  the  coupling  index 
D(JWp),  for  each  candidate  subset,  varies  from  0.01  to  2  percent  from  the  initial  to  desired  (or 
final)  view.  Thus,  calculation  of  D(JWp),  using  measurements  of  IJW^I  in  the  initial  view,  can 
be  used  to  evaluate  performance  of  the  different  candidate  feature  subset  selections. 

The  system  performance  is  evaluated  by  comparing  both  path  performance  and  time 
response.  It  was  shown  in  Section  2. 5. 2. 2,  that  if  J  is  constant,  the  predicted  path  (i.e.,  that 
which  occurs  if  perfect  model  following  were  achieved)  of  the  translational  stages  is  straight- 
line  motion.  For  the  examples  included  in  this  section,  changes  in  sensitivity  are  typically  less 
than  1  percent.  The  predicted  path  can  therefore  be  approximated  by  a  straight-line.  As 
D(JWp)  decreases,  we  anticipate  improved  model  following  capabilities  of  the  SISO 
controllers,  and  thus  closer  to  straight-line  motion.  Path  performance  will  be  evaluated  by 
comparing  D(JWp)  versus  the  actual  distance  traveled,  where  a  straight  line  path 
corresponds  to  0.144  inches.10  Time  domain  performance  is  evaluated  as  a  function  of  largest 
position  rise-time  of  the  three  axes. 

The  task  simulations  utilize  a  cube,  with  its  desired  image  displayed  in  Figure  7-1,  as  the 
control  object.  The  configuration  parameters  in  Table  7-1  are  defined  in  Figure  4-3. 


Focal  Length  F  =  35  mm 

a  =  45  degrees 
R  =  30  inches 
h  =  .5  inches 

Tobj  =  [-  5  0  0(in.),  0  0  0  (Euler-angles)] 

Desired  X-Y-0  stage  positions  =  -.5  (in),  0.  (in),  30.  (deg) 

Initial  X-Y  0  stage  positions  =  -.4  (in),  .1  (in),  31.  (deg) 

Table  7-1:  Small  Motion  Task  Configuration 

The  image  of  the  object  in  the  initial  view  is  not  displayed  since  it  appears  essentially  the  same 
as  in  the  desired  view. 


18,.  2  .  2.0.5 
(ax  +ay  ) 


=  0.144 


A  modification  of  the  aforementioned  configuration  will  be  used  to  discuss  a  case  for  which 
the  feature/joint  assignment  specified  in  (7.4)  does  not  hold.  In  this  example  (which  is 

dicussed  below),  the  cube  is  placed  on  one  of  its  edges  by  setting 
T«-  [•■5.0,0.0,46,0) 

The  camera  angle  and  position  are  modified  to  n  =  0  and  h  =  0.  The  remainder  of  the 
parameters  remain  as  in  Table  7-1.  The  image  of  the  cube  in  this  position  is  displayed  in 
Figure  7-2. 


Figu  re  7- 2 :  Cube  on  Edge 


Results  of  adaptive  control  simulations,  using  a  representative  set  of  feature  subsets,  are 
summarized  in  Table  7-2.  Discussion  of  these  results  will  follow  explanation  of  the  table 
entries.The  table  includes  the  feature  subset  and  the  performance  on  the  second  adaptive 
trial  using  that  subset.  It  was  verified,  for  each  trial,  that  the  actual  distance  traveled  are 
smooth  paths  and  do  not  oscillate  about  a  trajectory.  The  0-axis  trajectories  all  exhibited 
smooth  motions  with  negligible  overshoots  varying  between  0.01  to  0.4  percent.  Displays  of 
sample  trajectories  will  be  included  with  the  discussion.  The  table  indicates  the  coupling 
index  associated  with  each  feature  subset.  Additionally,  the  coupling  indices  for  each  of  the 
partitioned  (2x2)  matricies,  (7.5),  (7.6)  and  (7.7),  are  calculated.  The  maximum  value  for  each 
subset  is  indicated,  in  the  table,  by  D2x2.max-  The  significance  of  this  index  will  follow  in  the 
discussion  of  the  results.  Except  for  the  final  experiment,  all  examples  refer  to  the  image 
displayed  in  Figure  7-1.  These  17  candidate  feature  subsets  are  representative  of  the  108 
possible  combinations.  Of  these,  the  first  12  correspond  to  all  possible  combinations  if 
features  from  only  two  nodes  (i.e.,  nodes  1  and  4)  were  available.  The  remainder  are  examples 
of  combinations  using  features  from  all  three  nodes. 


Table  7-2:  Small  Motion  Task  Summary 

The  last  example,  which  uses  the  image  of  the  cube  on  its  edge  (Figure  7-2),  presents  a 
situation  for  which  the  feature/joint  assignment  specified  by  (7.4)  does  not  hold  for  all  of  the 
image’s  candidate  feature  subsets.  This  example  is  included  to  clarify  the  feature/joint 
assignment  issue.  In  this  case,  the  feature/joint  assignment  for  the  feature  subset  specified  in 
Table  7-2,  f  *  [X  ,,  A1t  Are|  4  t],  corresponds  to  (7.4).  However,  if  the  candidate  subset  had 
been  f  *  [XcogJ,  Ar  AfeM  5j,  then  the  sensitivity  becomes 

5X _ ,"1  [3.97E-3  2.91E-5  -1.78E-4]  5u" 

SAf  =  -1.30E-5  -3.78E-5  -1.50E-4  Su 
SA,eM.5  0.  1.69E-8  0.  I  «u' 

which  shows  that  the  relative  area  feature,  A  |-4  5,  must  be  coupled  to  the  Y-axis.  The  column 
organization  of  J,  which  minimizes  D(J  Wp),  for  this  case  is 

‘8XCo  i]  f3.97E-3  -1.78E-4  2.91E-51  Sux' 

SA,  s  -1.30E-5  -1.50E-4  -3.78E-5  Sue 
5Arel4  5  °-  °'  1.69E-8  J  5u 


The  important  observations  which  emerge  from  the  small-signal  evaluation  are: 

1.  Stability:  The  simulations  provide  further  verification  of  the  extension  of  SISO 
adaptive  controllers  to  achieve  stable  dynamic  control  of  coupled  systems.  The 
systems  are  stable,  with  acceptable  transient  responses,  and  zero  steady-state 
positional  errors.  For  example,  Figure  7-3,  shows  three  representative 
trajectories.  The  diverse  path  and  time  performances,  as  a  result  of  using 
different  feature  subsets,  emphasize  the  importance  of  feature  selection. 

2.  Feature  Selection:  The  coupling  index,  D,(JWp)  is  a  suitable  discriminant  function 
for  selecting  candidate  feature  subsets  to  improve  system  performance.  Plots  of 
coupling  index  versus  path  and  time  performance,  for  each  experiment  listed  in 
Table  7-2,  are  displayed  in  Figure  7-4.  Minimizing  D(JWp)  over  all  potential 
candidates  produces  a  fast,  straight-line  response.  The  feature  subset  which 
maximizes  D(JW  )  yields  the  slowest  response,  with  the  largest  path  deviation. 

As  D(JWp)  decreases,  both  path  and  time  performance  improve.  Local 
inconsistencies  are  observed,  with  non  significant  differences  in  performance. 

Order  of  magnitude  decreases  in  coupling  yield  substantial  performance 
improvements. 

Further  analysis  reveals  that  the  coupling  index  of  the  overall  sensitivity  matrix, 
D(JWp),  reflects  the  dominant  internal  couplings.  Plots  of  D2x2  max  are  displayed 
in  Figure  7-5.  Plots  of  D(JW  )  and  0o  ,  _  are  similar.  Using  either  coupling 
measures  would  have  predicted  the  same  selections.  The  significance  of  this 
observation  is  that  for  other  configurations  and  tasks,  one  of  these  measures  may 
turn  out  to  be  a  more  sensitive  discriminant  function.  Future  research 
investigations  should  include  evaluations  based  on  both  measures  to  determine 
which  is  ultimately  more  significant. 

The  small-signal  simulations  and  evaluations  show  the  importance  of  feature  selection,  and 
the  suitability  of  the  coupling  index  to  accomplish  this.  The  following  section,  which  includes 
examples  requiring  larger  motions  of  travel,  extends  these  results  to  tasks  for  which  variations 
in  J  are  significantly  larger. 


Figure  7*4:  Coupling  Index  vs.  Performance 
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Figure  7-5:  Partial  Coupling  Index  vs.  Performance 


7.4.  Large  Motion  Evaluation 

The  small-signal  simulations  provide  a  convenient  means  for  analysis  and  evaluation  of  the 
suitability  of  minimizing  D(JWp)  to  select  features.  In  practice,  actual  tasks  may  require  larger 
motions  which  result  in  time-varying  sensitivities.  Significant  changes  in  D(JWp)  can 
therefore  occur,  from  the  initial  to  the  final  view.  Selecting  features  based  on  minimizing 
D(JWp)  in  the  initial  view  may  not  assure  that  these  are  the  best  selections  along  the 
trajectory.  The  alternative  is  to  estimate  D(J  Wp)  on-line  and  then  implement  an  on-line  feature 
reselection  strategy  (i.e.,  analogous  to  the  feature/joint  reassignment  procedure  in  Chapters). 
This  becomes  a  more  complex  strategy.  The  evaluation  included  in  this  section  uses 
examples  of  large  motion  tasks  to  investigate  the  suitability  of  selecting  features  based  only 
on  coupling  in  the  initial  view.  This  section  also  includes  examples  of  the  fixed  and 
hierarchical  control  strategies. 


The  evaluation  uses  examples  based  on  two  task  configurations.  One  configuration,  which 
uses  the  cube  as  the  control  object,  is  represented  by  the  images  displayed  in  Figure  7-6.  This 
task  requires  net  trajectory  displacements  of  AX  =  2.5  in.,  AY  =  2  in.,  and  A0  =  3O  degrees. 
These  magnitudes  are  typical  of  those  that  observed  in  the  Flexible  Assembly 
Laboratory  [Sanderson  83b]  when  the  3  DOF  configuration  is  used  in  a  static  "look  and 


Although  such  a  large  trajectory  (i.e.  AY  =  20  in.)  is  not  anticipated  for  applications  with  this 
type  of  configuration,  it  is  useful  to  show  examples  which  further  accentuate  sensitivity  and 
coupling  changes. 


The  evaluation  does  not  include  calculation  of  the  predicted  path,  since  compact  analytic 
equations  for  the  feature  transformations  become  complex  to  derive  for  higher  DOF  systems 
with  solid  objects.  Future  investigations  could  derive  predicted  paths  by  numerical 
integration  methods,  using  estimates  of  J  measured  along  small  increments.  For  the 
examples  presented,  it  is  anticipated  that,  as  D(JWp)  decreases,  there  will  be  relative 
improvements  in  time  response,  and  path  deviations  from  a  monotonic  trajectory  will  be 
reduced.  That  is,  if  J  were  diagonal  (i.e.,  D=  -oo)  and  a  single  valued  function  of  position 
along  the  trajectory  (i.e.,  J.^0),  then  for  critically  damped  features,  the  path  must  be 
monotonic.  A  summary  of  system  performance  for  the  two  tasks  is  shown  in  Table  7-3.  The 
table  lists  performances  of  both  the  initial  and  second  (i.e. .subsequent)  adaptive  trials,  and  for 
fixed  and  hierarchical  control.  The  fixed  controller  is  implemented  by  fixing  the  adjustable 
gains  identified  in  the  first  adaptive  trial.  The  hierarchical  performance  is  indicated  for  the 
second  adaptive  hierarchical  trial.  Sample  plots  for  two  cases  are  displayed  in  Figures 
7-8  and  7-9. 
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Table  7-3:  Large  Motion  Task  Performance  Summary 

Comparison  of  the  coupling  indices  for  the  first  two  cases,  A  and  B,  reveals  that  the  feature 
subset,  which  minimizes  the  coupling,  can  change  from  the  initial  to  the  desired  views.  If  the 
feature  selections  were  limited  to  these  two  candidates,  then  an  on-line  feature  reselection 
strategy  may  be  useful.  A  completely  generalized  image-based  system,  using  SISO 
controllers,  may  therefore  require  this  capability.  For  cases  A  and  B  the  magnitudes  of 
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Figure  7-8:  3  DOF  Trajectory  (Case  D) 
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Figure  7-9:  3  DOF  Trajectory  (Case  F) 

D(JWp)  are  relatively  large  in  all  views.  If  more  potential  candidates  are  available,  then  the 
possibility  of  finding  a  subset  with  relatively  small  coupling  values,  over  the  trajectory, 
increases.  If  D(JWp)  remains  small,  then  reselection  strategies  may  not  be  crucial.  The  cases 
listed  do  not  produce  examples  with  both  small  couplings  and  predicted  reselections.  These 
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evaluations  do  suggest  that  feature  subsets,  which  have  initial  view  couplings  that  are  an 
order  of  magnitude  smaller  than  for  other  candidates,  show  significant  improvements  in 
performance. 

The  performance  features  for  each  control  strategy  are  summarized  as: 

1 .  Adaptive  Control:  Adaptive  control  yields  stable  system  performance  when  the 
object  stays  in  the  field  of  view.  The  cube  goes  OVF  in  both  the  first  and  second 
trials  of  Case  A.  This  feature  subset  has  relatively  large  values  of  coupling.  For 
Case  C,  where  the  coupling  is  smaller,  the  cube  goes  OVF  only  in  the  first  trial, 
but  with  improved  second  trial  response. 

The  controller  which  uses  the  feature  subset  with  the  smallest  initial  view 
coupling  (i.e.,  Case  D  for  first  task,  and  Case  F  for  second  task),  produces  the 
best  performance  for  each  task.  They  exhibit  the  fastest  rise-times  and  the 
shortest  distances  of  travel.  Their  trajectories  also  approach  straight-line 
motions.  The  deviations  from  a  straight  path  are  on  the  same  order  of  magnitude 
observed  for  the  2  DOF  linear  kinematic  configuration.  Further,  their  first  and 
second  adaptive  trial  performances  are  virtually  identical.  The  cases  with  larger 
coupling  show  significant  changes  between  these  two  trials. 

Rise-times  associated  with  the  second  task  (i.e.,  pyramid  examples)  are  much 
larger  due  to  control  signal  saturation  in  the  Y-axis  controller.  This  also  affects 
the  initial  part  of  the  trajectory  and  may  partly  account  for  the  absence  of  a 
straighter  path  there. 

These  examples  reinforce  the  suitability  for  using  D(JWp)  as  a  feature  selection 
discriminant  function.  These  results  also  suggest  that  the  selection  may  be  based 
on  the  initial  view  coupling,  if  potential  candidates  have  relatively  small  coupling 
values  (e.g.,  D(JWp)<0.) 

2.  Hierarchical  Control:  This  strategy  produces  system  performances  exhibiting  two 
important  characteristics.  First,  the  path  is  constrained,  for  each  case,  regardless 
of  the  magnitude  of  D(JWp).  After  most  of  the  relative  X-axis  motion  is  achieved, 
the  turntable  begins  the  orientation  phase.  Then,  after  most  of  the  orienting 
motion  is  achieved,  the  depth  control  begins.  This  strategy  prevented  the  object 
from  going  OVF. 

Second,  both  the  time  and  path  performances  of  each  case  are  virtually  identical. 

This  characteristic  does  not  result  from  the  trajectories  being  forced  to  pass 
through  regions  with  reduced  coupling.  It  was  verified  that  the  coupling  values,  at 
the  positions  where  each  controller  becomes  active  in  the  hierarchy,  are  not 
significantly  smaller  than  in  the  initial  or  desired  views.  This  strategy  does 
however  tend  to  cause  changes  in  any  feature  to  be  dominated  by  only  one  DOF. 

That  is,  as  each  controller  becomes  active,  the  majority  of  motion  for  the 
previously  activated  controller  has  already  taken  place.  This  facilitates  the  SISO 
identification. 
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3.  Fixed  Control:  Fixed  control  performance  is  not  significantly  different  than  for  the 
adaptive  approach.  However,  the  larger  motion  task  begins  to  accentuate  fixed 
controller  inadequacy  for  larger  gain  changes.  For  example,  the  rotational  stage 
response  for  case  F  exhibits  large  overshoot  (Figure  7-9). 

Fixed  control  of  the  smaller  motion  task  is  adequate  since  the  sensitivity  changes 
associated  with  the  these  cases  are  relatively  small.  For  example,  in  Case  D,  the 
percentage  change  of  dXcpg/DX  is  only  4  percent  from  the  initial  to  the  final  view, 
in  comparision  with  the  40  percent  change  for  the  2  DOF  example  represented  in 
Figure  6-6.  The  reduced  gains  for  the  3  DOF  configuration  result  from  larger 
focal  lengths  and  larger  relative  depths  between  the  camera  and  object. 
Considering  the  relatively  small  gain  changes  and  large  closed-loop  bandwidths, 
it  is  not  surprising  that  the  fixed  and  adaptive  responses  are  similar.  For  the  larger 
motion  pyramid  task,  for  which  gain  changes  are  an  order  of  magnitude  larger 
than  for  the  cube  task,  fixed  performance  begins  to  degrade. 

7.5.  Summary 

The  3  DOF  evaluation  provides  further  verification  of  the  extension  of  SISO  adaptive  control 
to  achieve  stablity,  with  acceptable  transient  response,  of  feature  coupled  systems.  The 
coupling  index.  D(JWp),  is  demonstrated  to  be  a  suitable  discriminant  function  for  selecting 
among  several  candidate  feature  subsets.  The  evaluation  shows  that  significant 
improvements  are  achieved  in  both  time  and  path  performance  as  D(JWp)  decreases.  For 
small-signal  tasks,  the  predicted  path  is  straight- line.  For  larger  motion  examples,  which  have 
time- varying  sensitivities,  the  observed  paths  approached  straight- lines  as  D(J  Wp)  decreased. 
The  generality  of  achieving  straight-line  paths,  for  other  tasks  and  configurations,  remains  to 
be  evaluated  in  future  research. 

The  hierarchical  approach  not  only  constrained  the  path,  but  also  facilitated  the  use  of  SISO 
controllers  for  even  large  coupling  values.  Thus,  the  hierarchical  approach  becomes  a 
conservative  and  safe  strategy  for  IBVS  implementations. 

Classical  control  theory  does  not  provide  an  effective  means  to  synthesize  fixed  SISO 
controllers  for  coupled  and  time-varying  systems.  The  adaptive  approach  provides  a  means 
for  synthesizing  the  fixed  SISO  controllers.  Fixed  controllers  achieved  performances  similar 
to  the  adaptive  approach.  As  gain  changes  become  larger,  fixed  performance  begins  to 
degrade. 


Chapter  8 
Conclusions 

8.1 .  Overview 

In  this  chapter,  the  author  summarizes  the  research  and  contributions  of  the  dissertation. 
Future  directions  for  research  are  suggested,  including  MIMO  control,  implementation  issues 
(including  higher  DOF),  and  comparative  evaluation  of  position  versus  image  based  control 
approaches.  Preliminary  investigations  into  some  of  these  areas  are  described. 

8.2.  Conclusions 

This  dissertation  provides  insight  and  analytic  tools  for  the  analysis,  design,  and  evaluation 
of  vision  based  dynamic  robot  control  systems.  These  goals  are  achieved  by: 

1.  Organizing  and  categorizing  previously  implemented  and  proposed  visual  servo 
control  systems  into  well-defined  classical  control  structures;  arid, 

2.  Analyzing,  designing,  and  evaluating  a  control  system  for  the  image  based 
approach. 

The  control  requirements  for  the  IBVS  "teach-by-showing"  strategy  (including 
nonlinearities,  coupling,  unknown  parameters,  delay,  noise,  saturation,  path  control,  and 
feature  selection)  are  analyzed  in  Chapter  2.  An  independent  MRAC  adaptive  controller,  a 
coupling  index,  and  a  path  constraint  hierarchy  are  proposed  to  satisfy  these  requirements. 

The  SISO  enhanced  SP  MRAC  adaptive  controller  is  derived  for  systems  with  delay  in 
Chapter  3.  Fixed  controllers  are  synthesized  by  fixing  the  adjustable  gains  of  the  adaptive 
controller.  Guidelines  for  selection  of  controller  parameters  are  provided.  Fixed  versus 
adaptive  control  is  evaluated  (in  Chapters  5  through  7),  using  computer  simulations,  for 
systems  with  increasing  complexity  to  determine  the  relative  contributions  (to  system 
performance  characteristics)  of  delay,  noise,  kinematic  and  dynamic  coupling,  and  feature 
coupling. 


SISO  adaptive  control  of  coupled  nonlinear  IBVS  systems  is  shown  in  simulation  to  be 
stable,  with  acceptable  transient  response,  and  zero  steady-state  errors.  System 
performance  (including  time  response  and  path  deviations)  improve  when  features  are 
selected  and  assigned  by  minimizing  the  coupling  index  D.  When  the  feature  sensitivities  are 
constant  and  coupled,  the  predicted  position  trajectories  are  straight- lines.  The  predicted 
trajectories  for  the  2  DOF  configurations  are  either  straight-line  or  approach  straight- lines. 
For  the  3  DOF  configuration,  the  observed  trajectories  approach  straight-line  as  the  coupling 
index  decreases.  The  generality  of  such  observations  must  be  evaluated  in  future  research. 
Deviations  from  the  predicted  paths  are  acceptable  for  both  small  and  large  motion  tasks, 
which  use  a  robot  with  linear  uncoupled  kinematics.  For  large  motion  tasks,  with  an 
articulated  arm,  kinematic  coupling  leads  to  unacceptahly  large  path  deviations.  The 
hierarchical  path  constraint  strategy  is  used  to  guide  the  robot  in  a  predictable  fashion  in 
Cartesian  space  (i.e. ,  moving  first  in  a  centering  X  direction,  then  moving  in  the  depth  Y 
direction).  The  generality  of  this  hierarchical  approach  remains  a  topic  for  future  research. 

Since  path  cannot  be  controlled  directly,  with  the  "teach-by-showing"  IBVS  approach,  the 
author  recommends  that  IBVS  can  be  used  for  tasks  for  which  exact  path  is  not  critical  (e.g.  in 
precision  assembly,  for  parts  acquisition  and  mating),  and  could  also  be  useful  to  increase  the 
accuracy  of  position-based  approaches. 

Fixed  LMFC  controllers  are  stable,  with  acceptable  transient  response.  Comparative 
evaluation  of  fixed  vs.  adaptive  control  shows: 

1.  A  fixed  controller,  tuned  for  one  task,  may  not  be  suitable  for  another  task.  A 
single  adaptive  controller  is  suitable  for  a  range  of  tasks. 

2.  Fixed  controllers  are  suitable  for  tasks  with  small  sensitivity  (or  gain)  changes 
(e.g.,  small  motion  tasks).  Adaptive  controllers  are  capable  of  tracking  large 
sensitivity  changes. 

3.  Fixed  control  exhibits  superior  noise  performance,  and  superior  stability  at  lower 
sampling-to-bandwidth  ratios.  Fixed  control  performance  becomes  sluggish  as 
the  specified  bandwidth  decreases. 

The  contributions  of  the  dissertation  are: 

1 .  Formalization  of  dynamic  visual  servo  control  of  robots,  by  organizing  and 
categorizing  them  into  well-defined  classical  feedback  control  structures.  These 
structures  are  classified  as  either  position-based  or  image  based,  and  control 
either  open-loop  robots  or  incremental  kinematic  robot  systems  (with  internal 
joint-level  compensation,  and  kinematic  decoupling).  The  formalization  facilitates 


the  control  system  design  by  showing  the  role  of  computer  vision  as  a  feedback 
transducer  (thus  affecting  closed-loop  system  dynamics,  and  requiring  a  visual 
servo  controller),  and  stating  the  static  and  dynamic  characteristics  (of  the  robot 
and  vision)  which  must  be  compensated.  The  categorization,  of  different  visual 
control  approaches,  provides  a  common  framework  for  future  comparative 
evaluations. 

2.  Introduction  and  demonstration  (by  simulation)  of  a  novel  "teach-by-showing" 
image-based  approach.  In  this  approach,  the  visual  servoing  task  is  specified  by 
taking  a  picture  of  an  object  in  the  desired  relative  position,  and  extracting  the 
image  features  in  this  view.  Explicit  knowledge  of  object  positions,  relationships 
between  features  and  positions,  and  robot  dynamics  are  not  required.  This 
simplified  approach  could  facilitate  the  integration  of  computer  vision  into  factory 
environments,  and  increase  feedback  accuracy  (required  for  precision  assembly) 
by  eliminating  inaccuracies  associated  with  feature  interpretation. 

3.  To  the  best  of  the  author's  knowledge,  the  analysis,  design,  and  evaluation  of 
IBVS  is  the  first  in-dopth  study  of  dynamic  robot  control  using  visual  feedback. 
Previous  studies,  appearing  in  literature,  have  been  limited  to  ad-hoc  control  or 
single  DOF  systems. 

4.  The  important  contributions  resulting  from  the  IBVS  analysis,  design,  and 
evaluation  are: 

a.  The  extension  of  an  enhanced  SP  MRAC  to  control  of  systems  with 
measurement  delay  and  nonlinear  gains.  For  such  systems,  the  delay 
requires  a  control  penalty  enhancement  to  achieve  acceptable  stability. 
The  control  penalty  is  also  required  to  assure  a  locally  stable  controller, 
since  the  hyperstable  identifier  can  identify  non-minimum  phase  zeros 
resulting  from  the  nonlinear  gains.  Reference  signal  biasing  is  required  to 
drive  steady-state  full-parallel  errors  to  zero,  since  the  identifier  can  identify 
type  0  plants  when  controlling  type  I  plants  with  nonlinear  gains. 
Sufficiently  large  sampling-to-bandwidth  ratios  are  required  to  track 
nonlinear  gain  changes. 

b.  To  the  best  of  the  author’s  knowledge,  the  dissertation  includes  the  first 
demonstration  (by  simulation)  of  the  enhanced  SP  MRAC  for  direct  joint- 
level  control  of  a  robot  arm.  Similar  algorithms  have  appeared  in  literature 
to  control  simulations  of  nonlinear  coupled  robot  dynamics,  but  have  not 
utilized  the  enhancements  and  stability  viewpoints  required  for 
implementation  issues  of  saturation,  noise,  and  delay. 

c.  Identification  of  limitations  and  applicability  of  SISO  control  of  coupled 
systems,  including: 

•  Introduction  of  an  index  to  measure  system  coupling.  The  ability  of 
the  SISO  controller  to  track  the  reference  signals  degrade  (and  path 
deviations  and  rise-times  increase)  with  increasing  values  of  the 
index.  This  index  is  shown  to  be  a  suitable  discriminate  function  for 
selecting  and  assigning  features  for  control. 
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•  Identification  of  a  sign -sensitivity  (Section  6.2.3. 1),  which  degrades 
transient  response.  Large  sampiing-to-bandwidth  ratios  are  required 
to  minimize  this  sensitivity. 

•  Observation  that  steady-state  identification  and  tracking  errors 
approach  zero,  but  path  deviations,  for  IBVS,  and  tracking  errors,  for 
joint-level  robot  control,  will  always  occur  with  SISO  contiol  of 
coupled  systems.  Transient  errors  are  reduced  (but  not  driven  to 
zero)  by  increasing  the  sampling -to  bandwidth  ratio. 

d.  Demonstration  of  the  adaptive  approach  as  a  means  to  synthesize  fixed 
controllers  for  nonlinear  and  coupled  systems.  Fixed  controller  design  is  a 
formidable  engineering  task,  for  such  systems. 

e.  Preliminary  analysis  of  the  relationships  between  path  trajectory  and 
feature  based  control.  This  analysis  encourages  future  evaluation  of  more 
general  systems  by  showing  that  (for  critically  damped  reference  feature 
signals,  each  with  the  same  time  constant): 

•  For  constant  and  coupled  feature  sensitivity  (i.e.,  Jacobian  of  the 
features  with  respect  to  positions),  the  predicted  paths  are  straight 
lines,  irrespective  of  the  number  of  DOF. 

•  For  the  2  DOF  line-in-space  configuration  tasks  (for  which  the  feature 
sensitivities  are  not  constant  and  change  dramatically  over  the 
trajectories),  the  predicted  paths  are  straight  lines  or  approach 
straight  line  motion. 

•  For  the  3  DOF  configuration,  the  observed  path  deviations  from  a 
straight  line  decrease  as  features  are  selected  to  reduce  the  coupling 
index. 

Using  a  path  constraint  hierarchy,  path  motion  can  be  constrained  along 
selected  DOF,  but  the  generality  of  this  approach  remains  a  topic  for  future 
research. 


The  image-based  "teach-by  showing”  approach  is  analogous  to  the  basic  robot  control 
"teach-mode"  operation.  In  the  author's  experience,  the  "teach-mode"  strategy  for  robot  task 
definition  has  greatly  facilitated  the  incorporation  of  robots  into  the  industrial  environment  by 
providing  a  simple  means  for  task  training.  Similarly,  the  author  hopes  that  the  "teach-by 
showing”  approach  will  have  the  potential  of  simplifying  applications  of  visual  servoing. 
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8.3.  Suggestions  for  Future  Research 

The  purpose  of  this  section  is  to  suggest  directions  for  future  research,  including  MIMO 
control,  physical  implementation  (including  higher  DOF),  and  comparative  evaluation  of  the 
position  and  image  based  approaches.  Preliminary  investigations  into  some  of  these  issues 
are  also  described. 

8.3.1 .  MIMO  vs.  SISO  Adaptive  Control 

The  dissertation  isolates  problems  of  SISO  control  of  coupled  systems,  including  sign- 
sensitivity,  feature/joint  assignment,  and  transient  tracking  errors.  An  MIMO  controller  may 
not  exhibit  similar  problems,  and  may  reduce  transient  tracking  errors.  Analysis,  design,  and 
evaluation  of  MIMO  control  is  a  fruitful  area  for  future  investigations.  While  the  major 
implementation  trade  off  between  MIMO  and  SISO  control  is  computational  complexity,  the 
author  believes  that  a  basic  problem  of  MIMO  control  will  be  the  additional  complexity  of 
selecting  and  initializing  the  controller  parameters,  which  must  be  empirically  chosen.  In  this 
section,  an  enhanced  MIMO  MRAC  controller  is  proposed,  and  related  issues  of  on-line 
estimation  of  J  and  JWp  (for  SISO  feature/joint  assignment)  are  described. 

The  mathematical  foundation  for  an  adaptive  MIMO  controller  can  be  found  in  [Borison  79], 
and  parallels  the  development  of  the  SISO  controller  in  Chapter  3.  For  MIMO  control,  the 
model  of  the  plant  under  control,  described  by  the  (m  x  m)  transfer  matrix  Hfz'1),  with  nd 
output  measurement  delays,  is 

Xd(z-,).z,ndH(z-1)U(z-1) 

The  system,  Hfz1),  is  identified  by  applying  the  hyperstable  adjustment  mechanism  [(3.4), 
(3.5),  and  (3.6)]  m  times  to  estimate  m  parameter  vectors.  For  MIMO  control,  the  dimension  of 
the  information  vector  and  parameter  vectors  increases  to  (2nm  x  1).  The  information  vector 
and  each  parameter  vector  are  used  to  predict  each  ouput  [Borison  79].  Calculation  of  the 
control  signals  require  matrix  inversion  operations  to  invert  the  vector  of  identified  outputs. 
And,  calculation  of  the  control  penalties  requires  evaluation  of  the  characteristic  polynomial 
of  an  inverted  matrix  of  estimated  gains. 

The  computational  complexity  of  MIMO  versus  m  independent  SISO  controllers  is 
summarized  in  Figure  8-1.  The  purpose  of  the  comparison  is  to  show  that  MIMO  control 
requires  more  computations  than  the  SISO  approach.  For  MIMO  control,  the  computational 


complexity  of  the  adjustment  mechanism  increases  by  a  factor  of  m  as  a  result  of  the 
increased  length  of  the  information  and  parameter  vectors.  The  number  of  computations 
indicated  in  Figure  8- 1  for  matrix  operations  is  a  conservative  estimate,  and  does  not  account 
for  the  possibility  of  more  efficient  matrix  manipulation  algorithms.  The  salient  feature  is  that 
complexity  of  MIMO  control  is  order  m3n2,  versus  mn2  for  SISO  control.  As  computer 
hardware  becomes  faster  and  less  expensive,  the  computational  differences  between  these 
approaches  will  become  less  significant.  The  SISO  approach  may  still  remain  attractive  for 
factory  environments  due  to  its  modularity. 
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Figu  re  8- 1 :  MIMO  vs.  SISO  Computational  Complexity 


8.3.1 . 1 .  On-Line  Sensitivity  Estimation 


For  linear  kinematic  robot  configurations,  it  appears  that  off-line  measurement  of  JWp  (prior 
to  the  start  of  control)  is  sufficient  for  calculation  of  the  coupling  D,  to  select  and  assign 
features.  General  implementations  of  SISO  control,  of  coupled  systems,  will  require  on  line 
estimation  of  JWp  for  selecting  feature/joint  assignment.  In  the  dissertation,  a  a  low- 
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frequency  assumption  is  made  to  evaluate  Wp(jw)  (for  the  purpose  of  calculating  the  coupling 
index  D).  Since  the  low-frequency  robot  gains  can  be  measured  off  line,  only  J  need  be 
estimated  on-line. 

The  author  suggests  that  either  J  or  JWp  can  be  measured  on-line  using  the  MIMO 
hyperstable  adjustment  mechanism,  described  in  Section  8.3.1.,  with  the  parameter  matrix 
(i.e.,  the  matrix  of  the  m  estimated  parameter  vectors:  replaced  with  or 

/3*-JW  .  For  estimation  of  J,  the  information  vector  becomes  the  robot  joint  displacement 
changes  (n  :<  1),  and  jij  is  (n  x  1).  For  estimation  of  JWp,  the  information  and 

parameter  vectors  remain  (2nm  x  1). 

8.3.2.  Implementation 

The  evaluation  of  computer  simulated  IBVS  control  (Chapters  5,  6,  and  7)  demonstrates  the 
feasibility  of  this  approach,  and  encourages  hardware  implementations.  The  3  DOF 
configuration  (in  Chapter  7)  can  be  implemented  with  hardware  currently  available  in  the 
Robotics  Institute  Flexible  Assembly  Laboratory  [Sanderson  83b].  The  author  is  currently 
developing  hardware  and  software  to  implement  an  automated  PC  board  insertion  task 
(depicted  in  Figure  1-2),  using  IBVS  control.  To  image  the  lead  tip,  a  specially  configured 
fiber  optic  sensor  [Agrawal  83]  is  being  fabricated  (to  replace  the  CCD  camera),  to  achieve 
high  sampling  rates,  high  resolution,  and  low  noise,  .  This  system  will  use  a  3  DOF  Cartesian 
robot. 

IBVS  implementations,  using  real  robots  and  sensors,  raise  control  issues  relating  to  robot 
mechanical  structure,  sensor  resolution,  and  extension  of  IBVS  control  to  higher  DOF.  These 
issues  are  discussed  below. 

8. 3. 2.1 .  Robot  Mechanical  Structure 

Physical  characteristics  of  the  robot  (and  actuators),  which  may  place  additional  constraints 
on  the  sampling  period  selection  (Section  3.6.2),  include  structural  resonances,  dynamics 
which  are  characterized  by  complex  poles,  and  torque  off  sets  required  to  overcome  static 
friction  at  low  velocities. 

It  has  been  suggested  [Paul  81]  that  to  avoid  resonant  excitation  of  non  rigid  structures 
characterized  by  a  structural  resonant  frequency  uQ,  the  sampling  period  should  be 
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If  the  robot  dynamics  exhibit  complex  poles  at  «±jwmax,  the  sampling  period  of  the  MRAC 
discrete  time  hyperstable  identifier  is  required,  by  the  Nyquist  sampling  theorem,  to  be 

m 

T  < - . 

40  max 

If  the  vision  system  cannot  extract  features  at  a  rate  fast  enough  to  satisfy  the  resonance  and 
Nyquist  constraints,  the  author  suggests  augmenting  the  IBVS  control  with  joint- level  velocity 
feedback  controllers,  which  would  sample  at  higher  rates  (e.g.,  using  tachometer  feedback) 
than  the  main  IBVS  feedback  loop.  The  velocity  controllers  (which  could  be  adaptive)  would 
be  tuned  to  dampen  structural  resonances  and  place  the  poles  of  the  minor  loop  feedback  on 
the  real  axis,  to  satisfy  the  Nyquist  constraints.  High-gain  velocity  feedback  overcomes  torque 
off-sets  (or  equivalently  actuator  voltage  off  set)  by  amplifying  ihe  small  velocity  errors. 

8-3. 2. 2.  Sensor  Resolution 

Spatial  resolution  of  the  sensor  (e.g.,  discrete  pixel  quantization  of  a  CCD  camera)  is 
manifested  as  measurement  noise.  Low-pass  filtering  of  the  measurement  signals  can  reduce 
the  noise  amplitude.  The  author  suggests  the  following  strategy  to  further  reduce  these 
effects.  When  the  spatial  resolution  of  the  sensor  (denoted  by  R)  is  less  than  the  distance 
traveled  during  a  sampling  interval  (i.e.,  R<Vre(T),  then  fix  the  MRAC  adjustable  gains  (at  their 
current  value)  to  create  a  fixed  LMFC  controller,  since  the  LMFC  is  less  sensitive  to  noise. 

8. 3. 2. 3.  Extension  to  Higher  DOF 

The  2  and  3  DOF  configurations,  evaluated  in  this  dissertation,  are  practical  for  specific 
tasks.  Implementation  of  more  general  systems  (e.g.,  with  other  features  and  5  or  6  DOF) 
main  to  be  analyzed  for  the  relationships  of  feature  control  to  world  coordinate  paths,  and 
evaluated  for  the  extension  of  SISO  control  to  higher  DOF, 

At  the  outset  of  this  research,  the  author  conducted  preliminary  experiments  of  IBVS  control 
using  simulations  of  a  5  DOF  articulated  arm.  While  this  system  was  not  investigated  in-depth, 
it  served  to  demonstrate  IBVS  feasibility,  and  to  gain  insight  into  the  control  issues  which  are 
addressed  in  this  dissertation.  The  system  consisted  of: 

1. A  5  DOF  articulated  arm  (with  kinematic  structure  similar  to  the  Cincinnati- 
Milicron  T3),  modeled  with  constant  linear  uncoupled  dynamics. 

2.  A  camera  (as  modeled  in  Section  4.5)  is  mounted  to  the  last  joint  (or  end- 
effector). 
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3.  The  task  is  to  position  the  end-effector  relative  to  polyhedral  objects  (Section 
4.5).  Extracted  features  include  both  areas  and  centroids. 

4.  The  controller  consists  of  5  SISO  fixed  proportional  controllers,  which  are 
manually  tuned. 

5.  Feature  selection  and  assignment,  and  path  constraint  hierarchies  were  specified 
using  empirically  (i.e.,  no  coupling  index  was  used). 

The  controllers  could  be  tuned  (using  empirical  gain  adjustments  and  feature  selections)  to 
achieve  stability,  with  acceptable  transient  response  and  zero  steady-state  positional  errors. 
The  Cartesian  paths  were  not  derived  (i.e.,  only  the  joint  motions,  and  initial  and  final  relative 
positions  were  monitored).  The  proportional  gains  suitable  for  one  task  were  not  acceptable 
for  another  task.  It  was  difficult  to  tune  the  system  for  large  motion  trajectories  (i.e.,  the 
response  was  sluggish  for  low  proportional  gains,  oscillatory  for  high  gains,  and  it  was 
difficult  to  keep  the  object  in  the  field  of  view.)  Feature/joint  assignments  and  the  path 
constraint  hierarchy  had  to  changed  with  varying  robot  arm  configurations. 

8. 3. 2. 4.  Position  vs.  Image  Based  Control 

Approaches  to  visual  servo  control  are  categorized  (in  Chapter  2)  into  either  position  or 
image  based  structures.  The  organization  of  visual  servoing  into  well  defined  classical 
control  structures  provides  a  framework  for  designing  and  comparing  the  different  controt 
approaches.  An  analysis,  design,  and  evaluation  of  position  based  systems,  and  their 
comparative  evaluation  with  image  based  control,  would  provide  a  complete  set  of  design 
tools  for  implementation  of  dynamic  visual  servo  control  systems. 

Since  each  approach  will  have  tradeoffs,  the  author  suggests  that  they  may  be  used  to 
augment  one  another.  For  example,  position  based  approaches  have  the  advantage  of 
directly  controlling  path,  while  inaccuracies  of  the  required  interpretation  (i.e.,  of  features  to 
world-space  coordinates)  decrease  closed-loop  accuracy.  Augmenting  position  based 
control  with  IBVS,  by 

1.  Deriving  the  control  signal  by  adding  both  position  based  and  image  based 
control  signals;  or 

2.  Initially  controlling  gross  positioning  with  position- based  control;  then  switching 
over  to  IBVS  for  fast  and  accurate  fine  control 

seem  to  be  natural  combinations. 

The  author  hopes  that  the  proposed  research  and  implementations  will  enable  IBVS  control 
to  become  a  practical  system  for  robot  control. 
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Appendix  A 

MRAC  Without  Measurement  Delay 

The  purpose  of  this  appendix  is  to  explain  and  present  a  concise  derivation  of  the  MRAC 
controller  developed  by  Neuman  and  Morris  in  [Neuman  80]. 

A  block  diagram  for  a  SISO  process  under  discrete  computer  MRAC  control  is  shown  in 
Figure  3-1  [Neuman  80],  An  MRAC  system  comprises  three  basic  components: 

1.  Analog  plant  under  control; 

2.  Full-parallel  reference  model;  and 

3.  Adaptive  controller 

The  reference  model  transfer  function  Mfz'1)  specifies  the  desired  closed-loop  response  of 
the  sampled  plant  output,  y(k),  to  a  reference  signal  r(k).  The  adaptive  controller  is  a  digital 
feedback  controller.  The  controller  gains  are  adjusted  to  drive  the  closed-loop  response  of 
the  system  to  that  of  the  reference  model.  The  controller  uses  both  plant  and  model  input  and 
output  information  to  generate  the  control  signal  u(k). 

At  the  heart  of  the  MRAC  is  the  controller  adjustment  mechanism.  If  the  full-parallel  (FP) 
output  error  is  defined  as 

eofc)  =  x(k)-y(k),  (A.l) 

the  adjustment  mechanism  must  be  designed  so  that 

1 .  e^k)  asymptotically  approaches  zero,  and 

2.  The  closed-loop  system  is  stable. 

Adjustable  controller  design  is  based  on  the  identification-error  method  [Neuman  80].  The 
plant  input-output  information  generates  estimates  of  the  plant  parameters  and  are  used  by 
an  identifier  to  predict  the  plant  output.  The  identification  error  is 

e|0(k)  =  y(k)-t(k)  (A.2) 


t  \ 
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where  the  identifier  output  t(k)  is  the  predicted  plant  output.  This  error  drives  the  adjustment 
mechanism  which  updates  the  estimates  of  the  plant  parameters.  In  turn,  these  estimates  are 
used  to  adjust  the  controller  gains.  The  adjustment  mechanism  is  designed  so  that  e|D(k)  is 
globally  asymptotically  stable  for  a  SISO  linear  system.  The  controller  is  then  designed  to 
guarantee  the  asymptotic  stability  of  the  full-parallel  error. 

A.1 .  Reference  Model 

The  discrete  time  reference  model  should  generate  a  realizable  output,  x(k),  for  an  analog 
process  under  computer  control.  Thus,  the  model  can  be  specified  by  the  cascade  of  a 
digital-to-analog  converter  (DAC),  an  analog  process,  and  a  analog-to-digital  converter  (ADC) 
with  the  desired  response  characteristics. 

If  the  analog  process  model  is  m,h  order,  with  m1  zeros  (rr^cm),  the  discrete  model  transfer 
function  is  [Neuman  79a] 

.  Rfz'1)  B  (z-1)  B  (z'1) 

M(z'  )  = - ~=  ,n  ,  =■■  -  (A.3) 

X(z-1)  Am(z  ')  1-AV) 

b  °z  *  + . . .  +  b  °  z*m 


1-a  z  *,..a  z 

I  m 

The  sampled  data  FP  model  output  can  thus  be  characterized  by  the  m,h  order  difference 
equation 

m  m 

xFp(k>=[X]  bjV'jr(k)+ ®|V]  x(k)  (A.5) 

i  =  1  is  1 

where  the  delay  operator  q'1  is  defined  by 

(q-')x(k)±x(k.i). 

For  example, 

CE  bjq']r(k)  =  b1r(k-1)  +  b2r(k-2)  +  ...  +  bmr(k-m). 

is  1 

The  model  is  a  full-parallel  model  since  (A.3)  is  recursive. 

Reference  model  structures  have  typically  been  specified  by  critically  damped,  second 
order  systems  [Landau  79]  for  a  wide  variety  of  adaptive  control  tasks.  The  closed-loop  model 
can  be  generated  by  assuming  that  a  type  I  second-order  linear  plant,  G^^fs),  is  under 
computer  control  (Figure  A-1),  and  the  proportional  gain  Kpcrjt  is  tuned  to  provide  critical 
damping.  The  open-loop  plant  transfer  function  is 
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Figure  A-1:  FP  Reference  Model 

Neuman  and  Morris  specified  the  reference  model  in  the  Z-plane.  The  derivation  presented 
here  differs  by  first  showing  an  analog  plant  model  in  a  feedback  control  structure.  This 
approach  facilitates  development  of  a  reference  model  when  measurement  delays  are 
included  (in  Chapter  3). 

The  equivalent  z-transform  model  of  Figure  A-1  is  shown  in  Figure  A-2. 


Figure  A-2:  Discrete  Reference  Model 
In  Figure  A-2,  the  digital  transfer  function 
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(z'1)  =  [1  z  ')*Z {  [ L  1  (  — )  ] t<_kT} 


model 


models  the  cascade  of  the  DAC,  analog  process  G(s),  and  ADC  [Neuman  79a], 
second-order  type  I  analog  model  in  (A.6), 
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where,  for  the  sampling  period  T, 


at  =  1  +  e'T/Tmodel 
a2  =  -e  r/Tmodel 

bl  =  ^KTmodel^1  T/Tmodel  ®  fT^ 


b2  =  (Kr^  J(1  -e  T/Tmodel-e  rT  T/rmnrt<ll) 


model7 

'model^'  '•*  ' — "5,'w  "•model' 

The  closed-loop  transfer  function  of  Figure  A-2  is 
-i;  KPon.lbi2'1+b2z'21 


WCL^Z  I  -  .  Ilf  U  1,1 


.2 


1  +  lKPcritbrailZ  +  lKPcritb2  +  &2^Z 
For  critical  damping,  the  gain  term  Kpcrt(  must  satisfy 

lKPcritbraJ  =  4lKPcritb2  +  a2l 
The  difference  equation  of  the  closed-loop  model  becomes 

xFp(k)  =  KpCritbir(*<'1)  +  ^Pcntb2r^'^ 

+  ^ai’KPcritb1^X^k1^  +  ^a2kPcritb2^X^k'^ 

and,  the  model  parameters  for  (A. 5)  are 

bT=biKPcrit 

b2  =  b2KPcrit 
ai  =ai"b1KPcrit 
a2  =  arb2KPcrit 


m  =  2. 


For  the 

(A. 7) 
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A. 2.  Plant  Model  Structure 

For  the  identifier  to  track  the  plant  output,  the  structure  of  the  plant  must  be  defined.  If  the 
plant  under  control  is  assumed  to  have  an  equivalent  nth  order  I/O  structure  (Section  2.3.1 .3), 
with  nt  zeros  (n^n),  then  the  corresponding  discrete-time  model  is  the  n,h  order  transfer 
function 

.  Y(z-1)  B  (z'1) 

W(z'  )  = - P  ,  (A. 8) 

U(z'  )  Ap(z'  ) 

b,z  1  +  b2z  2  +  ...  +  bnz  n 
~  1*a1z'1-a2z‘2- . -anzn 


where  the  {a^  and  {bj  are  unknown  parameters.  The  sampled  plant  output,  y(k),  can 
therefore  be  modeled  by  the  nlh  order  input-output  difference  equation 

n  n 

y(k)  =  [£  bjO  ’]  u(k)  +  a.q ’]y(k).  (A.9) 

i=1  i  =  1 

In  vector  notation,  (A.9)  is 

y(k)  =  £Ti!>(k-1) 


where 


£'=(b1...bna1...an)  is  the  ( 2n  X  1)  parameter  vector;  and 
4>(k-1)=[u(k-1)...u(k-n)y(k-1)...y(k-n)]T  is  the  (2n  X  1)  information  vector 


(A- 10) 


A. 3.  Controller 

A  series-parallel  (SP)  identifier  with  adjustable  parameters  tracks  the  plant  output.  The 
identifier  output  t(k)  is  calculated  according  to 


n  A  n 

t(k)=[X]  b^k-Ug^uM*^  aj(k-1)qi]y(k) 


(A.11) 


where  {b;}  and  {a^  are  estimates  of  the  process  parameters  of  an  equivalent  SISO  linear 
system. 

The  identifier  is  series-parallel  since  it  is  non- recursive.  A  full  parallel  identifier  produces 
unbiased  estimates  of  the  process  parameters  in  the  presence  of  noise  (assuming  a  control 
signal  sufficiently  rich  in  frequency  content),  but  at  a  slower  speed  of  convergence  than  for  a 

A 

SP  adjustable  model.  In  addition,  an  FP  identifier  requires  good  initial  guesses  for  {b^  and 


A 

{aj.  In  contrast,  the  SP  identifier  produces  biased  estimates  of  the  plant  parameters.  In 
MRAC  control,  it  is  shown  below,  all  that  is  required  for  FP  errors  to  approach  zero 
asymptotically  is  an  eciuivalent  input/output  model  which  matches  the  input/output 
characteristics  of  the  plant. 

In  vector  notation,  (A.1 1)  becomes 

t(k)  =  /£T(k-1)l&(k-1) 
at 

where  £  (k- 1 )  is  the  estimated  parameter  vector.  The  identification  error  becomes 
e|D(k)  =  y(k)-t(k)  =  [£T-£T  (k- 1 )  ]  <!>(k- 1 ). 

In  the  next  section,  an  adjustment  mechanism  is  presented  to  guarantee  asymptotic  stability 
of  e|D(k). 

When  the  identification  error  converges  to  zero,  the  identifier  becomes  an  equivalent  input- 
output  model  of  the  plant.  The  adaptive  controller  is  then  constructed  by  solving  for  the 
control  signal  u(k)  required  to  force  the  identifier  output  t(k  +  1)  to  follow  the  reference  signal 
XR(k).  By  inverting  (A.1 1),  the  control  signal  becomes 

1  n  A  n  A 

u(k)  =  jv— (xR(k)-[^  b.(k)q-<M)]u<k)-[£  ai(k)q<M>]y(k)}  (A.12) 

bi(k)  i=2  i  =  1 

It  is  now  shown  that  true  estimates  of  the  plant  parameters  are  not  required  to  produce  an 
asymptotically  stable  FP  error.  True  estimates  may  not  occur  when: 

1 .  Parameter  estimates  become  biased  in  the  presence  of  noise; 

2.  The  frequency  content  of  the  control  signal  is  poor;  and 

3.  As  demonstrated  in  this  dissertation,  when  the  plant  being  identified  is  nonlinear, 
time-varying,  or  part  of  a  coupled  system. 

If  the  reference  model  signal  is  chosen  to  be  the  output  of  the  one-step  ahead  FP  model, 

XR(k)  =  x(k  +  1)  =  [V;  b:q(M,]r(k)+[£  a(q<M)]x(k),  (A.13) 

i  =  t  i=1. 

then,  upon  convergence  of  e|D,  the  identifier  becomes 
t(k  +  1)  =  XR(k)  =  x(k  +  1) 
or, 

t(k)  =  x(k), 


and  the  FP  error  is 
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e^  =  x(k)-y(k) 

=  t(k)-y(k) 

=  -e|D(k)  (A.  14) 

Thus,  the  FP  output  error  is  asymptotically  stable  if  the  adjustment  mechanism  can  assure 
asymptotic  stability  of  the  identification  error,  regardless  of  errors  in  the  true  parameter 
estimates! 

A. 4.  Adjustment  Mechanism 

A 

The  adjustment  mechanism  used  for  estimating  /?  can  be  designed  from  either  parametric 
optimization  or  stability  viewpoints.  Optimization  techniques,  which  are  based  on  cost- 
function  minimization,  were  first  applied  to  adaptive  control  [Whitaker  61],  These  include  the 
gradient  descent  and  least-squares  methods.  As  applied  to  adaptive  control,  these  methods 
have  drawbacks  which  require  reliable  initial  estimates  of  the  plant  parameters  [Price  70]  and 
result  in  low  adaptation  speeds  to  guarantee  local  stability  [Kokotovic  66],  In  contrast,  the 
stability  viewpoint,  which  is  inherent  to  the  design  goals  of  closed-loop  control  systems, 
guarantees  asymptotic  stability  of  e|D  and  thus  e^at  higher  adaptation  speeds,  regardless  of 
initial  parameter  estimates.  This  approach,  which  includes  Lyapanov’s  second 
method  [Narendra  74]  and  Popov’s  hyperstability  theorem  [Popov  63],  prescribe  stability 
criteria  for  nonlinear,  time-varying  ,  closed-loop  systems.  The  output  e)D  of  the  nonlinear, 
time-varying,  closed-loop  adjustment  mechanism  drives  the  parameter  update,  and  is 
designed  to  be  asymptotically  stable.  Landau,  who  was  the  first  to  apply  hyperstability  to 
MRAC  [Landau  69],  claims  that  with  respect  to  the  Lyapanov  approach  "....  More  general 
results  were  obtained  using  the  hyperstability  approach,  perhaps,  because  it  is  more 
systematic."  [Landau  74] 

The  hyperstable  adjustment  mechanism  [Landau  79]  specifies  the  following  identifier  when 
the  plant  is  assumed  to  be  linear  with  slowly  varying  parameters: 


A  a  1 


£(k)  =  &{k- 1 )  +  -P(k- 1  )<£(k- 1  )s(k) 

A 

(A. 15) 

e_(k) 

s  k  = - ^ - 

1  +  (1  /A)4>T(k-1)P(k-1)<f>(k- 1) 

(A.  16) 

1  1  r  s(k)  q  T 

P(k)  =  -P(k- 1 )  -72L— -  J  P(k- 1  )«T»(k- 1  )£T(k- 1  )P(k- 1 ) 

A  A  ejD'k) 

(A. 17) 

where  P(k)  is  a  (2n  X  2n)  adaptive  gain  matrix,  s(k)  is  the  a' posteriori  error  (which  is  the  error 
signal  after  the  parameters  are  updated  at  the  kth  computational  cycle),  and  e(D  is  the  a’priori 
error  signal.  The  fading  factor  X  (0<A<1)  weighs  past  values  of  the  input/output  samples  by 
the  progression 

1,  X,  X2 . Ak. 

If  X  =  1 ,  new  information  is  averaged  with  all  past  data,  minimizing  overreaction  to 
measurement  noise.  If  X <1 ,  old  data  is  weighed  less,  and  the  mechanism  can  track  the 
parameter  H  even  when  the  parameters  are  slowly  time-varying. 

The  adjustment  mechanism  in  (A. 15)  -  (A. 17)  approximates  the  optimal  recursive  weighted 
least-squares  algorithm  which  is  derived  from  the  minimization  of  the  cost  function 

k 

J(k)-(«/2)i;  A'e|D2(k-i) 
i  =  0 

This  approach  is  referred  to  as  the  hyperstable  least-squares  adjustment  mechanism. 

A. 5.  Enhancements 

Performance  of  the  basic  MR  AC  algorithm  can  be  significantly  improved  by  the  addition  of 
numerous  enhancement  strategies  [Neuman  79b).  Neuman  and  Morris  present  a  detailed 
analysis  of  enhancement  mechanisms  [Morris  79],  and  recommend  an  enhanced  error 
identification  MRAC  algorithm.  These  enhancements  include: 

1.  Control  Penalty; 

2.  Series-Parallel  Reference  Model  Signal; 

3.  Augmenting  Error  Filter;  and 

4.  Auxiliary  Fixed  Control. 

The  augmenting  error  filter  must  be  used  with  caution  since  it  increases  the  high-frequency 
gain  of  the  feedback  controller,  thus  amplifying  noise  and  reducing  the  stability  margin.  To 
use  the  auxiliary  fixed  controller  requires  that  the  system  operate  around  a  known  and 
stationary  operating  point,  which  in  general  does  not  hold  for  IBVS  systems.  Thus,  only  the 
first  two  enhancements,  which  are  described  below,  are  used  in  this  dissertation. 

Control  Penalty 

A  control  signal  penalty  is  obtained  by  multiplying  (A.  12)  by  the  positive  scaler  w(k) 
(0<w<1).  This  penalty  is  introduced  to 


243 


1.  Reduce  the  magnitude  of  the  control  signal  and  thereby  the  effect  of  reduced 
marginal  stability  due  to  control  signal  saturation; 

2.  Reduce  control  signal  noise;  and 

3.  Insure  a  stable  and  bounded  control  signal. 

Experimental  evaluation  of  MRAC,  by  Neuman  and  Morris,  demonstrates  that  control  signal 
saturation  in  the  DAC  can  occur  quite  often  when  controlling  low- pass  systems,  such  as  robot 
joint  actuators.  The  feedback  path  of  the  FP  controller  can  be  characterized  as  a  high-pass 
filter  when  the  process  under  control  exhibits  a  low- pass  frequency  response.  Measurement 
noise  is  amplified  and  the  control  signal  quality  deteriorates. 

Guaranteeing  the  asymptotic  stability  of  e(D  does  not  insure  the  asymptotic  stability  of  eQFP 
when  a  control  penalty  is  used. The  identification  error  becomes 

fp  b.(k) 

e|D(k)  =  -eo  W  +  — 1 — (l-vz)u(k-l) 

7 7 

If  ejD(k)  goes  to  zero,  the  FP  error  is  related  to  the  control  signal  according  to 
fp  b.(k) 

eoFP  =  -J— (l-Tr)u(k-l)  (A. 18) 

7 1 

If  the  control  signal  becomes  unbounded,  the  FP  error  is  not  asymptotically  stable.  This  can 
occur  if  the  actual  or  estimated  plant  has  non-minimum  phase  zeros  lying  outside  the  unit 
circle.  The  computations  below  illustrate  that  the  poles  of  the  adjustable  controller  are 
determined  by  the  estimated  plant  zeros.  Thus,  when  the  zeros  lie  outside  of  the  unit  circle, 
the  controller  poles  become  unstable.  In  addition,  the  control  signal  can  become  quite  large  if 
the  plant  zeros  are  not  accurately  estimated.  This  often  occurs  in  actual  implementations 

A 

when  b  t  is  underestimated  due  to  measurement  and  signal  transmision  delays  through  the 
ADC  and  DAC  converters. 

A 

To  specify  the  control  penalty,  a  lower  bound  bLB  is  placed  on  the  magnitude  of  b^k)  used 
to  calculate  the  controller  gain.  The  value  of  w(k)  is  specified  according  to 
b.(k)  a 

b,(k)<bL8 

°LB 

1  if  b.j(k)>bLB. 

It  remains  to  choose  bLB  so  that  u(k)  remains  bounded.  To  accomplish  this  task,  the  Jury 


stability  conditions  are  applied  to  test  the  location  of  the  controller  poles  (Pj)  at  each 
computational  cycle.  When  pole  magnitudes  exceed  the  design  parameter  y,  where  0<y<1, 
the  value  of  bLB  is  increased  (thus  reducing  w)  until  the  poles  lie  within  a  circle  of  radius  y  in 
the  Z-plane. 

The  controller  characteristic  equation  is  determined  by  taking  the  Z-transform  of  (A. 12)  with 
the  control  penalty  enhancement  included.  The  control  signal  is 


-{xR(z  ')-[V  aj(k)z  *M)]  V(z' 1) } 


and  the  controller  characteristic  equation  is 
F(z1)=  1+w - V  b(k)z(M). 

bi(k)^  ,V 

Thus,  the  estimated  plant  zeros  determine  the  controller  poles. 


(A.  19) 


The  Jury  conditions  are  applied  to  (A. 19)  for  a  second-order  system  (n  =  2)  by  expressing 
the  characteristic  equation  in  positive  powers  of  z  as: 


b„(k) 

F(z)  =  z  +  w(k)*-2 - . 


(A.20) 


To  test  for  lpl<y,  apply  the  transformation  z  =  yz  to  obtain 

b„(k) 

F(z)  =  yz  +i(k)^ - . 

b^k) 

For  this  characteristic  equation,  the  Jury  condition  is: 


IwMb^kJ/^OOKy 


and  the  control  penalty  is  calculated  according  to 


Ib/^Ky 


w(k)  = 


(A. 21) 


(b,/b2)y  if  lb2/b,l>y 


Series-Parallel  Reference  Model  Signal 
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Neuman  and  Morris  have  shown  that  increasing  the  control  penalty  of  the  FP  MRAC  (to 
suppress  noise  and  increase  the  stability  margin)  can  lead  to  a  highly  oscillatory  closed-loop 
response.  An  enhancement  which  further  reduces  the  high-frequency  gain  of  the  feedback 
path  (thus  reducing  noise  amplification  and  producing  a  closed-loop  system  which  does  not 
exhibit  a  highly  oscillatory  response  as  the  control  penalty  is  increased)  is  the  series-parallel 
(SP)  reference  model  signal  enhancement.  The  SP  reference  model  signal,  which  improves 
performance  at  the  expense  of  slower  tracking  of  the  FP  model,  is  implemented  by  replacing 
the  recursive  component  x(k)  of  (A.  13)  by  a  non-recursive  filtering  of  the  process  output  y(k): 

X>)  =  [E  biq(,'1)]r(k)  +  [X]  afi'(,‘1)]y(k)-  (A. 22) 

i=i  c=i 

In  contrast  to  the  control  penalty,  this  enhancement  leads  to  an  asymptotically  stable  FP 
output  error 


°  A(q') 


(A. 23) 


where 

If  the  FP  model  specifies  the  stable  filter  £  l/A^q'1)]  ,  the  FP  output  error  is  asymptotically 
stable  if  e)D(k)is  asymptotically  stable.  Transient  identification  errors  produce  slower  tracking 
of  the  reference  model  due  to  this  filtering  effect. 


Appendix  B 

Dynamic  Equations  for  2  DOF  Arm 


This  appendix  presents  the  derivation  of  the  dynamic  equations  of  motion  for  the  2  DOF  arm 
described  in  Section  (4.4).  The  derivation  is  based  on  the  Lagrange- Euler  formulation  for  rigid 
body  structures  [Paul  81].  The  Lagrangian  function  L  is  first  defined  as  the  difference 
between  the  kinetic  energy  K  and  the  potential  energy  P  of  the  system: 

L=K-P 

The  dynamic  equations  are  then 
3  3L  3L 

Fi  =  r-T^~r—  (B-1) 

1  at  aqj  aq( 

where  qj  and  q(  are  the  generalized  position  and  velocity  coordinates.  F.  is  the  corresponding 
force  or  torque  acting  on  the  i,h  joint.  For  an  articulated  robot  arm,  q.ifT,  and  F.  corresponds 
to  joint  torque.  If  an  n  DOF  manipulator  is  described  by  a  set  of  homogeneous  transforms 
{Aj},  then  (B.1)  becomes 

Fi  =  E  E  TRACE(— ij  — i)q 
fn  k= i  d%  1 0oi  k 


j=1  k  =  1 

n  i  i 


^  j  I  Ji2t  OfT 

(_3J 3T.) 

m  t  k  m 


j  =  1  k  =  1  m= 1 

A  lEi  H 
£  m'9  *  ' 


where 


T  —A  A  A 
k~1  2  -  Hk 

The  pseudo- inertial  matrix,  Jj,  of  the  i!h  link  is: 
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1 


-I.  +1.  + 1 

IXX  IVV 


IVV _ !£ 

j  i. 

1. 

ml. 

2 

ixy 

1.  *1.  +1. 

ixx  ivv  iz 

IXZ 

z  I 

1  1 

1. 

ixy 

2 

iyz 

1.  +1.  -1. 

nvy-j 

1. 

1. 

IXX  IVV  IZZ 

m.z 

1  1 

IXZ 

lyz 

2 

mx. 

m.y'i 

m.z. 

m. 

* 

where  the  first  and  second  moments  are 


'xx =  /(y2  +  z2)dm  :  •yy  =  J (*2  +  z2)dm  :  'zj  =  /(x2  +  y2)dm ; 

•xy  =  /xy dm ;  |xz  =  /xzdm;  iyz  =  /yzdm; 

mx  =  y  x  dm  ;  my  =  J y  dm  ;  mz  =  f  z  dm  m  =  J  dm 


and  1  r*  is  the  homogeneous  vector  defining  the  position  of  the  center  of  mass  of  link  j  relative 
to  the  jth  coordinate  frame,  g  defines  the  gravitational  field.  And,  m.  is  the  mass  of  the  ith  link. 


For  n  =  2,  (B.2)  is: 


F)={TRACE(^lj)iIi)  +TRACE(^J2 
3q,  3q,  3q,  2 

♦Ctrace(5j>§)K 

*{trace(sS;j'S)*trace(^ 

+  (trace (-^2- J  +  TRACe(  — 

'3q,3q2  23q,'  V3q, 

+  {TrAce(AI2_j  p)}q  2 
3q23q2  2  3q, 

T  1  -»•  T  3Tg  2->. 

m  ,g  T- -  r  m  g  —2  2r 

3q,  3q,  2 


F2={TRACE(^2j2^2)q,}  +  (tRACE^ 
3q,  3q2  < 

.{trace(-^J2^T)}^ 

3q,3q1  2 3q2 

+  (trace (-ii-j  £L)}  +  ( TRACE ( 
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do2dO^ 


dox dd2 


dll  ~  12  ”12  '  '  2  — 12 

~~  =  "S.™  -C  3  0  -r  S  2 

a/}  12  12  2  12 

ou2au2  0000 


3T i  ”12  ”12  _  '  2  — 12  '  1~1 

- —  =  s„  -C,0  0  •r„S,,-r,S1 

?)/)  ar?  12  ,2  2  12  i  i 

d6,id'/i  0  0  0  0 


The  Jj  and  J2  matricies  are  calculated  for  the  links  in  Figure  B-2  and  Figure  B- 1. 


Figure  B- 1 :  Link  ft 2 


Figure  B-2:  Link  #  1 

Since  the  each  link  consists  of  a  point  mass,  located  at  the  orgin  of  it's  coordinate  frame,  and 
a  thin  rod  in  the  Xdirection,  all  of  the  interial  components  in  the  Y  and  Z  directions  are  zero. 


0  0 


(mr,  +  mpi  +  mc) 


riL  g  +mc'1  +2k'-l 


The  gravitational  components  are  defined  next.  If  gravity  acts  in  the  X  direction,  then 


g  =  [g  0  0  0]T 


To  find  the  center  of  mass  of  the  second  link,  r  ,  set  the  first  moment.  /  xdm,  equal  to  the 

^-6C|  J 


net  mass  times  r„  .  Thus, 
2-eq 


mr?ro 

-  2  =  (mr2  +  mP2>W 


m 


r2 


2  eq  “  2(mM  +  mp2)  ^ 


and  hence 


V  ['*«,  0  0  1JT. 


Similarly  for  link  1 


,r  rai+m(1+l)] 

h-  o  c '  2k ' J 


1-eq 


(m  ,  +  m  ,  +  m  )r, 

'  rl  pi  c'  1-eq 

and  hence 

'^-[r,.a0  0  1]T 


Equations  (B.3)  and  (B.4)  become: 


F1  “  D11^1  +  D  12^2  +  D1 12^1^2  +  D122^22  +  D1 


Fi  =  D,i0,+Doo0o  +  Do1i012  +  D^ 


oj  eg 


Appendix  C 

2  DOF  Controller  Parameters 


The  independent  SP  MRAC  controller,  for  the  2  DOF  configurations,  is  outlined  in  this 
appendix.  The  same  controller  parameters  are  used  for  both  the  linear  and  nonlinear 
kinematic  configurations.  All  simulations  and  control  structures  include  a  measurement  delay 
(i.e.,  nd=1).  Independent  adaptive  and  fixed  controllers  are  implemented  in  Chapter  6 
according  to: 

1.  Control  signal :  (3.10) 

2.  Identifier :  (3.4), (3.5),  and  (3.6)  with  ^=0.85  .  Tjje  identifier  is  initialized  according 
to  Table  C-1,  and  the  initial  signs  of  b,(0)  and  b2(0)  are  selected  according  to  the 
sign  of  the  corresponding  diagonal  elements  of  the  sensitivity  matrix.  The  values 
in  Table  C- 1  are  identical  to  those  used  in  the  single  degree  of  freedom  examples 
(in  Table  5-1),  with  an  exception  at  the  sampling  period  T  =  0.003  sec.  The  gain 
matrix  initilization  is  increased,  to  yield  acceptable  response  for  the  nonlinear 
robot  configuration. 

3.  Control  penalty  :  (3. 12),  with  y  =  .65 

4.  SP  Reference  Model :  (A. 22).  The  reference  model  bandwidth  ,«n,  is  specified  for 
each  set  of  examples,  while  T  is  varied  to  evaluate  the  effects  of  changing  the 
sampling-to-bandwidth  ratio  (fs/fBW).  The  open-loop  reference  model  time 
constants  are  selected  according  to  the  methods  outlined  in  section  3.6.1.,  and 
the  discrete  model  parameters  are  generated  according  to  the  method  outlined  in 
Section  3.3.3.  The  same  closed-loop  bandwidth  is  specified  for  each 
independent  controller. 

5.  Scale  Factors : 

SFt=  033  =  2500 
SFT  =  . 013  =  5000 
SFT  =  .0O3  =  10000 

6.  LMFC  :  The  fixed  controller  is  implemented  by  fixing  the  gains  of  the  adjustable 
controller  to  the  staedy-state  values  identified  in  initial  adaptive  trials. 


Appendix  D 

3  DOF  Controller  Parameters 


The  simulations  in  Chapter  3  are  implemented  with  the  sampling  period  T  =  0.003  (sec)  and 
a  bandwidth  «n  =  44.51  (sec'1),  or  fs/fgw  =  47,  as  *n  the  2  DOF  evaluation.  The  MRAC 
controller  is  implemented  according  to  the  outline  presented  in  Section  6.2  with  the  exception 
of  the  identifier  initialization  and  scaling  factors.  The  identifier  is  initialized  according  to  Table 


T ( sec ) 


Xcog  &  A  re i  Feature 
Control  1 ers 
Diag  [.1  .1  .01  .01] 


Absolute  Area  Feature 
Control  1 ers 
Oiag  [.5  .5  .1  .1] 


P  Thresh 


[±1  ±1  1.5  -0.5] 


Table  D- 1 :  3  DOF  Parameter  Initialization  (T  =  0.003  sec) 

The  initial  adaptive  gains,  P(0),  for  the  absolute  area  identifiers  (i.e.,  A;  or  Aj  +  j)  are  larger  than 
for  the  other  features.  When  all  feature  identifiers  were  initialized  with  the  lower  gains, 
P(0)  =  [0.1  0.1  0.01  0.01],  the  area  controller  would  exhibit  poor  transient  response  and  result 

A  A 

in  sgn[b,  ^j^sgnfbg  ss].  It  was  observed  throughout  this  research  that,  when  this  occurs, 
improved  transient  responses  can  be  achieved  by  increasing  the  initial  adaptive  gains  for  that 
feature,  assuming  large  enough  sampling-to-bandwidth  ratios. 


The  initial  parameter  vector  gain  initializations  [bt(0)  and  b2(0)]  were  reduced  from  the 
values  used  for  2  DOF  configuration.  In  the  3  DOF  configuration,  the  camera  will  be  mounted 
at  a  relatively  long  distance  from  the  turntable,  which  significantly  reduces  the  magnitudes  of 
J,  and  therefore  plant  gains. 
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